I fit a model… now what?

About me

  • Québécois
  • PhD University of Michigan
  • Associate Professor of Political Science, Université de Montréal
    • Social Science Research Methods
    • Political Economy
  • First upload to CRAN: 2009-06-03

Problem 1: Inconsistency

Predicted probabilities with standard errors:

library(nnet)

# logit
mod1 <- glm(vs ~ hp + mpg, data = mtcars, family = binomial)

# Multinomial logit
mod2 <- multinom(factor(gear) ~ hp + am, data = mtcars, trace = FALSE)

pred1 <- predict(mod1, se.fit = TRUE, type = "response")
pred2 <- predict(mod2, se.fit = TRUE, type = "probs")

Problem 1: Inconsistency

Logit:

str(pred1)
List of 3
 $ fit           : Named num [1:32] 0.7034 0.7034 0.8841 0.7005 0.0227 ...
  ..- attr(*, "names")= chr [1:32] "Mazda RX4" "Mazda RX4 Wag" "Datsun 710" "Hornet 4 Drive" ...
 $ se.fit        : Named num [1:32] 0.1356 0.1356 0.0905 0.1336 0.0431 ...
  ..- attr(*, "names")= chr [1:32] "Mazda RX4" "Mazda RX4 Wag" "Datsun 710" "Hornet 4 Drive" ...
 $ residual.scale: num 1

Multinomial logit:

str(pred2)
 num [1:32, 1:3] 0.000194 0.000194 0.000107 0.557004 0.969816 ...
 - attr(*, "dimnames")=List of 2
  ..$ : chr [1:32] "Mazda RX4" "Mazda RX4 Wag" "Datsun 710" "Hornet 4 Drive" ...
  ..$ : chr [1:3] "3" "4" "5"

Problem 2: Interpretation

summary(mod1)

Call:
glm(formula = vs ~ hp + mpg, family = binomial, data = mtcars)

Deviance Residuals: 
     Min        1Q    Median        3Q       Max  
-2.09164  -0.19536  -0.01377   0.50499   1.18424  

Coefficients:
            Estimate Std. Error z value Pr(>|z|)  
(Intercept)  9.53119    7.03368   1.355   0.1754  
hp          -0.07234    0.03461  -2.090   0.0366 *
mpg         -0.03385    0.18097  -0.187   0.8516  
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

(Dispersion parameter for binomial family taken to be 1)

    Null deviance: 43.860  on 31  degrees of freedom
Residual deviance: 16.803  on 29  degrees of freedom
AIC: 22.803

Number of Fisher Scoring iterations: 7
summary(mod2)
Call:
multinom(formula = factor(gear) ~ hp + am, data = mtcars, trace = FALSE)

Coefficients:
  (Intercept)           hp        am
4    5.262061 -0.049919859  8.312471
5   -9.944043  0.005586538 16.891350

Std. Errors:
  (Intercept)         hp        am
4    3.206564 0.02694180 16.665212
5   17.956716 0.04888094  1.771992

Residual Deviance: 20.09822 
AIC: 32.09822 

What does “log odds” mean in the real world?

Solution: marginaleffects

Interpretation:

  • Post-estimation methods to interpret statistical models

Consistency:

  • 5 general purpose technologies
  • 68 classes of models supported

Alternatives (try them!)

emmeans

  • Most popular
  • Feature gap is closing (reversing?)
  • Fewer models supported
  • Feels complicated (to me!)

margins:

  • Author has recommended marginaleffects

modelbased and ggeffects

  • Easy
  • Great plots
  • Wrappers around emmeans
  • Less flexible

Try them; they’re great!

So why marginaleffects?

  • Simple
  • Powerful
  • Composable
  • Valid
  • More models
  • Active development (I have tenure and love this stuff.)

Five general purpose technologies

Interpret statistical models with:

  1. Delta method
  2. Adjusted predictions
  3. Contrasts
  4. Marginal effects
  5. Marginal means

Demo: 1892 free CSVs on RDatasets

https://vincentarelbundock.github.io/Rdatasets/

Fair’s Affairs (1969)

Number of extra-marital affairs, children, age, etc.

affairs <- read.csv("https://vincentarelbundock.github.io/Rdatasets/csv/AER/Affairs.csv")
tail(affairs)
       X affairs gender age yearsmarried children religiousness education
596 1935       7   male  47         15.0      yes             3        16
597 1938       1   male  22          1.5      yes             1        12
598 1941       7 female  32         10.0      yes             2        18
599 1954       2   male  32         10.0      yes             2        17
600 1959       2   male  22          7.0      yes             3        18
601 9010       1 female  32         15.0      yes             3        14
    occupation rating
596          4      2
597          2      5
598          5      4
599          6      5
600          6      2
601          1      5

Fair’s Affairs (1969)

Poisson model:

library(marginaleffects)
mod <- glm(
  affairs ~ children + gender + yearsmarried,
  family = poisson,
  data = affairs)
summary(mod)

Call:
glm(formula = affairs ~ children + gender + yearsmarried, family = poisson, 
    data = affairs)

Deviance Residuals: 
   Min      1Q  Median      3Q     Max  
-2.144  -1.769  -1.384  -1.169   6.647  

Coefficients:
              Estimate Std. Error z value Pr(>|z|)    
(Intercept)  -0.389798   0.092200  -4.228 2.36e-05 ***
childrenyes   0.038412   0.105335   0.365    0.715    
gendermale    0.030381   0.067653   0.449    0.653    
yearsmarried  0.076891   0.007682  10.009  < 2e-16 ***
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

(Dispersion parameter for poisson family taken to be 1)

    Null deviance: 2925.5  on 600  degrees of freedom
Residual deviance: 2766.5  on 597  degrees of freedom
AIC: 3268.5

Number of Fisher Scoring iterations: 6

Delta method

Standard errors for functions of parameters.

Delta method

The childrenyes and gendermale coefficients are close:

coef(mod)
 (Intercept)  childrenyes   gendermale yearsmarried 
 -0.38979791   0.03841220   0.03038081   0.07689095 

Tests of equality:

deltamethod(mod, hypothesis = "childrenyes = gendermale")
                      term    estimate std.error statistic   p.value   conf.low
1 childrenyes = gendermale 0.008031391 0.1263135  0.063583 0.9493023 -0.2395385
  conf.high
1 0.2556013
deltamethod(mod, hypothesis = "b2 = b3")
     term    estimate std.error statistic   p.value   conf.low conf.high
1 b2 = b3 0.008031391 0.1263135  0.063583 0.9493023 -0.2395385 0.2556013
deltamethod(mod, hypothesis = c(0, 1, -1, 0))
    term    estimate std.error statistic   p.value   conf.low conf.high
1 custom 0.008031391 0.1263135  0.063583 0.9493023 -0.2395385 0.2556013

Delta method

deltamethod(mod, hypothesis = "b2 * exp(b3) = 0.02")
                 term  estimate std.error statistic   p.value   conf.low
1 b2 * exp(b3) = 0.02 0.0195971 0.1085638 0.1805123 0.8567504 -0.1931841
  conf.high
1 0.2323783
  • Non-linear functions of the params
  • Standard errors everywhere
  • All marginaleffects functions have a hypothesis argument
    • Are two predictions/contrasts/mfx equal?
    • Cross-group contrasts

Adjusted Predictions

The outcome predicted by a fitted model on a specified scale for a given combination of values of the predictor variables, such as their observed values, their means, or factor levels (“reference grid”).

aka, in-sample predictions, fitted values.

One predicted value per observation, the usual way:

predict(mod, type = "response")
        1         2         3         4         5         6         7         8 
1.5060531 0.9210563 2.2299485 2.2987358 0.7395237 0.7599813 0.7173943 2.2987358 
        9        10        11        12        13        14        15        16 
2.2299485 0.7834244 2.2987358 0.9866488 2.2987358 0.7599813 0.9210563 2.2299485 
       17        18        19        20        21        22        23        24 
2.2299485 0.7173943 0.7599813 1.5181975 0.7599813 0.7599813 1.5181975 1.5181975 
       25        26        27        28        29        30        31        32 
0.9866488 0.7599813 1.1600217 2.2987358 0.9866488 0.9571244 2.2987358 0.7599813 
       33        34        35        36        37        38        39        40 
0.7208288 2.2299485 0.9866488 0.7599813 2.2299485 0.9210563 0.8141029 0.7173943 
       41        42        43        44        45        46        47        48 
1.5650294 2.2987358 0.6992588 0.9571244 1.2054476 0.9494681 1.2054476 2.2299485 
       49        50        51        52        53        54        55        56 
0.8141029 2.2987358 0.7173943 1.2426320 0.9866488 1.5650294 0.9494681 2.2299485 
       57        58        59        60        61        62        63        64 
0.8141029 2.2299485 0.7173943 1.5650294 2.2299485 1.5650294 0.7173943 0.9494681 
       65        66        67        68        69        70        71        72 
1.1958049 2.2987358 1.5650294 2.2299485 0.7599813 0.7599813 0.9210563 1.5181975 
       73        74        75        76        77        78        79        80 
0.6837339 2.2987358 2.2987358 1.2426320 0.7897418 0.9866488 0.7599813 2.2987358 
       81        82        83        84        85        86        87        88 
0.8141029 2.2987358 0.7173943 0.9494681 0.7897418 0.6992588 2.2299485 0.7599813 
       89        90        91        92        93        94        95        96 
0.7897418 2.2299485 2.2299485 1.5650294 1.5650294 2.2987358 0.7208288 2.2299485 
       97        98        99       100       101       102       103       104 
2.2987358 1.5650294 2.2987358 2.2987358 1.5181975 1.5650294 0.6837339 2.2987358 
      105       106       107       108       109       110       111       112 
2.2299485 0.9866488 1.2426320 2.2987358 2.2987358 0.7599813 1.2054476 2.2299485 
      113       114       115       116       117       118       119       120 
0.7599813 1.5650294 1.1600217 2.2121107 2.2987358 1.5060531 0.7173943 1.2054476 
      121       122       123       124       125       126       127       128 
1.2054476 2.2299485 2.2987358 1.2054476 0.7599813 2.2987358 0.6837339 0.7834244 
      129       130       131       132       133       134       135       136 
0.7834244 0.7834244 1.5181975 2.2987358 0.7599813 0.9494681 1.5181975 2.2299485 
      137       138       139       140       141       142       143       144 
0.7173943 2.2299485 0.9866488 0.9494681 1.5181975 2.2299485 1.2426320 2.2987358 
      145       146       147       148       149       150       151       152 
0.9494681 0.9866488 0.9866488 2.2299485 2.2299485 1.2426320 1.2426320 0.7395237 
      153       154       155       156       157       158       159       160 
0.9866488 2.2987358 0.7684831 2.2987358 2.2299485 2.2299485 1.2426320 0.9866488 
      161       162       163       164       165       166       167       168 
0.9571244 0.9571244 2.2987358 2.2299485 1.5181975 0.7897418 2.2299485 0.7599813 
      169       170       171       172       173       174       175       176 
2.2987358 2.2987358 2.2987358 0.6837339 1.5181975 2.2299485 0.7599813 0.7324256 
      177       178       179       180       181       182       183       184 
0.9210563 1.5181975 1.2054476 2.2299485 2.2299485 0.8141029 0.9571244 1.1600217 
      185       186       187       188       189       190       191       192 
0.7266414 1.2054476 0.7395237 0.9866488 1.5650294 2.2987358 0.7395237 1.2054476 
      193       194       195       196       197       198       199       200 
0.7490561 2.2987358 2.2299485 0.9866488 2.2987358 0.7599813 2.2987358 0.7599813 
      201       202       203       204       205       206       207       208 
0.9866488 0.9866488 0.7599813 0.7395237 1.2054476 0.7173943 2.2299485 0.7599813 
      209       210       211       212       213       214       215       216 
1.4609860 2.2299485 2.2299485 0.9494681 1.2426320 0.7395237 0.9210563 0.7834244 
      217       218       219       220       221       222       223       224 
0.7834244 0.9494681 0.9571244 2.2987358 1.5181975 2.2987358 2.2299485 0.7834244 
      225       226       227       228       229       230       231       232 
1.5181975 1.2054476 1.5181975 0.8141029 2.2987358 0.9866488 2.2299485 1.2054476 
      233       234       235       236       237       238       239       240 
2.2299485 0.9494681 0.7395237 0.9866488 0.7173943 2.2299485 1.5650294 2.2299485 
      241       242       243       244       245       246       247       248 
1.2426320 2.2299485 2.2987358 0.9866488 2.2299485 0.7684831 0.7599813 2.2299485 
      249       250       251       252       253       254       255       256 
2.2299485 2.2987358 2.2299485 2.2987358 1.2426320 1.2054476 0.7834244 0.9571244 
      257       258       259       260       261       262       263       264 
0.7599813 2.2987358 2.2987358 1.2054476 0.9571244 2.2987358 1.2426320 0.7599813 
      265       266       267       268       269       270       271       272 
0.7897418 2.2987358 2.2299485 1.5181975 2.2987358 0.9571244 2.2987358 0.7834244 
      273       274       275       276       277       278       279       280 
0.6837339 1.5650294 0.9210563 1.2054476 0.9866488 2.2299485 2.2121107 0.7834244 
      281       282       283       284       285       286       287       288 
2.2987358 2.2299485 1.2054476 2.2299485 1.2054476 0.7834244 0.7834244 0.9866488 
      289       290       291       292       293       294       295       296 
1.2054476 2.2299485 1.2054476 0.7834244 2.2987358 0.7599813 1.5181975 0.9494681 
      297       298       299       300       301       302       303       304 
0.6992588 0.9210563 2.2987358 1.5650294 1.2054476 0.9866488 0.9866488 0.7834244 
      305       306       307       308       309       310       311       312 
0.9571244 0.7173943 1.5650294 0.7173943 2.2987358 0.9494681 0.7599813 1.2054476 
      313       314       315       316       317       318       319       320 
2.2987358 0.9866488 2.2299485 0.9866488 0.9571244 2.2987358 0.7599813 2.2987358 
      321       322       323       324       325       326       327       328 
0.6992588 0.7599813 0.9866488 2.2299485 0.7599813 0.9494681 2.2987358 1.4609860 
      329       330       331       332       333       334       335       336 
1.5181975 0.8141029 0.7834244 1.5181975 2.2299485 1.5181975 0.9866488 2.2299485 
      337       338       339       340       341       342       343       344 
0.7173943 2.2299485 2.2299485 1.5181975 1.1958049 2.2299485 0.8141029 0.7454871 
      345       346       347       348       349       350       351       352 
0.9866488 0.7048250 0.8141029 1.2426320 2.2299485 0.7834244 0.7599813 0.7599813 
      353       354       355       356       357       358       359       360 
0.7048250 0.9571244 0.9571244 2.2299485 2.2299485 1.2426320 0.7395237 0.7048250 
      361       362       363       364       365       366       367       368 
1.5650294 0.6992588 2.2299485 1.5181975 2.2987358 0.9866488 1.2054476 1.5181975 
      369       370       371       372       373       374       375       376 
1.5181975 0.9210563 1.2054476 2.2987358 1.2426320 0.7599813 0.7834244 0.7897418 
      377       378       379       380       381       382       383       384 
1.2054476 2.2299485 0.7173943 0.7897418 0.9571244 2.2987358 1.5650294 2.2987358 
      385       386       387       388       389       390       391       392 
1.2054476 0.7599813 1.5181975 0.7173943 0.7599813 2.2299485 1.2054476 2.2987358 
      393       394       395       396       397       398       399       400 
2.2299485 2.2987358 1.2054476 0.9494681 0.7599813 0.7173943 2.2299485 0.7834244 
      401       402       403       404       405       406       407       408 
0.9866488 0.9571244 2.2987358 0.7834244 2.2121107 1.1958049 2.2987358 2.2987358 
      409       410       411       412       413       414       415       416 
1.1600217 1.2054476 0.9210563 1.2426320 1.1958049 1.5650294 0.7599813 0.9571244 
      417       418       419       420       421       422       423       424 
2.1459156 2.2987358 1.2054476 1.4609860 0.9866488 0.9571244 2.2299485 0.7599813 
      425       426       427       428       429       430       431       432 
1.2426320 0.9866488 0.7048250 1.2054476 2.2299485 1.5650294 2.2299485 0.7599813 
      433       434       435       436       437       438       439       440 
0.9866488 2.2299485 1.2054476 1.2054476 2.2299485 2.2299485 1.2426320 1.2426320 
      441       442       443       444       445       446       447       448 
2.2299485 0.9866488 1.2054476 0.7599813 0.9571244 0.9210563 0.7599813 2.2121107 
      449       450       451       452       453       454       455       456 
1.5650294 2.2987358 0.9571244 0.7834244 0.9571244 2.2987358 1.5181975 0.7048250 
      457       458       459       460       461       462       463       464 
0.7897418 2.2987358 0.7599813 2.2987358 2.2299485 2.2299485 2.2299485 2.2299485 
      465       466       467       468       469       470       471       472 
1.5650294 2.2299485 0.9494681 1.5650294 0.9210563 2.2987358 2.2299485 0.9571244 
      473       474       475       476       477       478       479       480 
1.2054476 0.8141029 1.2054476 2.2299485 1.5181975 0.7834244 0.9494681 1.2054476 
      481       482       483       484       485       486       487       488 
1.5181975 0.9866488 0.7173943 1.5181975 1.2054476 2.2987358 1.4609860 1.5181975 
      489       490       491       492       493       494       495       496 
2.2987358 2.2299485 2.2987358 2.2987358 0.9210563 1.2426320 0.9866488 2.2987358 
      497       498       499       500       501       502       503       504 
2.2299485 0.9494681 1.2054476 0.7490561 2.2987358 2.2987358 0.9866488 0.9494681 
      505       506       507       508       509       510       511       512 
2.2299485 0.9494681 1.5181975 1.2426320 1.2426320 0.7834244 0.9866488 2.2987358 
      513       514       515       516       517       518       519       520 
2.2299485 0.9571244 0.9866488 1.2426320 2.2987358 2.2299485 1.5650294 2.2987358 
      521       522       523       524       525       526       527       528 
2.2299485 1.2054476 1.2426320 0.9494681 1.2426320 2.2299485 2.2299485 1.1600217 
      529       530       531       532       533       534       535       536 
1.2054476 0.8141029 2.2299485 1.5650294 0.9866488 0.9571244 2.2299485 1.5650294 
      537       538       539       540       541       542       543       544 
1.5650294 1.2054476 1.2054476 1.5650294 0.7454871 2.2299485 1.1600217 1.2426320 
      545       546       547       548       549       550       551       552 
0.9866488 0.7897418 2.1459156 2.2299485 2.2987358 1.5060531 2.2299485 2.2987358 
      553       554       555       556       557       558       559       560 
2.2987358 2.2299485 2.2987358 2.2987358 1.5650294 2.2987358 2.2299485 1.5650294 
      561       562       563       564       565       566       567       568 
2.2987358 0.7599813 2.2299485 2.2987358 0.9210563 1.5181975 0.9571244 1.1958049 
      569       570       571       572       573       574       575       576 
0.9571244 2.2299485 0.9210563 0.7454871 2.2299485 0.9210563 0.9866488 0.7897418 
      577       578       579       580       581       582       583       584 
2.2299485 1.2426320 2.2121107 2.2987358 0.9866488 2.2987358 2.2987358 0.7834244 
      585       586       587       588       589       590       591       592 
0.9866488 2.2987358 2.2299485 2.2121107 2.2299485 2.2299485 1.2054476 2.2987358 
      593       594       595       596       597       598       599       600 
2.2299485 1.2426320 1.5650294 2.2987358 0.8141029 1.5181975 1.5650294 1.2426320 
      601 
2.2299485 

Fitted values for the original data

Simple data frame with broom naming convention:

predictions(mod)
    rowid     type predicted  std.error  conf.low conf.high affairs children
1       1 response 1.5060531 0.14741307 1.2431523 1.8245519       0       no
2       2 response 0.9210563 0.07913696 0.7783073 1.0899868       0       no
3       3 response 2.2299485 0.12427095 1.9992124 2.4873147       0      yes
4       4 response 2.2987358 0.12851060 2.0601684 2.5649293       0      yes
5       5 response 0.7395237 0.06835445 0.6169859 0.8863984       0       no
6       6 response 0.7599813 0.06737756 0.6387605 0.9042068       0       no
7       7 response 0.7173943 0.06475348 0.6010723 0.8562274       0       no
8       8 response 2.2987358 0.12851060 2.0601684 2.5649293       0      yes
9       9 response 2.2299485 0.12427095 1.9992124 2.4873147       0      yes
10     10 response 0.7834244 0.07116344 0.6556575 0.9360891       0       no
11     11 response 2.2987358 0.12851060 2.0601684 2.5649293       0      yes
12     12 response 0.9866488 0.07787951 0.8452290 1.1517303       0      yes
13     13 response 2.2987358 0.12851060 2.0601684 2.5649293       0      yes
14     14 response 0.7599813 0.06737756 0.6387605 0.9042068       0       no
15     15 response 0.9210563 0.07913696 0.7783073 1.0899868       0       no
16     16 response 2.2299485 0.12427095 1.9992124 2.4873147       0      yes
17     17 response 2.2299485 0.12427095 1.9992124 2.4873147       0      yes
18     18 response 0.7173943 0.06475348 0.6010723 0.8562274       0       no
19     19 response 0.7599813 0.06737756 0.6387605 0.9042068       0       no
20     20 response 1.5181975 0.07908880 1.3708372 1.6813985       0      yes
21     21 response 0.7599813 0.06737756 0.6387605 0.9042068       0       no
22     22 response 0.7599813 0.06737756 0.6387605 0.9042068       0       no
23     23 response 1.5181975 0.07908880 1.3708372 1.6813985       0      yes
24     24 response 1.5181975 0.07908880 1.3708372 1.6813985       0      yes
25     25 response 0.9866488 0.07787951 0.8452290 1.1517303       0      yes
26     26 response 0.7599813 0.06737756 0.6387605 0.9042068       0       no
27     27 response 1.1600217 0.10233572 0.9758300 1.3789804       0       no
28     28 response 2.2987358 0.12851060 2.0601684 2.5649293       0      yes
29     29 response 0.9866488 0.07787951 0.8452290 1.1517303       0      yes
30     30 response 0.9571244 0.07498949 0.8208765 1.1159865       0      yes
31     31 response 2.2987358 0.12851060 2.0601684 2.5649293       0      yes
32     32 response 0.7599813 0.06737756 0.6387605 0.9042068       0       no
33     33 response 0.7208288 0.06721263 0.6004310 0.8653687       0       no
34     34 response 2.2299485 0.12427095 1.9992124 2.4873147       0      yes
35     35 response 0.9866488 0.07787951 0.8452290 1.1517303       0      yes
36     36 response 0.7599813 0.06737756 0.6387605 0.9042068       0       no
37     37 response 2.2299485 0.12427095 1.9992124 2.4873147       0      yes
38     38 response 0.9210563 0.07913696 0.7783073 1.0899868       0       no
39     39 response 0.8141029 0.07692276 0.6764741 0.9797323       0      yes
40     40 response 0.7173943 0.06475348 0.6010723 0.8562274       0       no
41     41 response 1.5650294 0.08231414 1.4117337 1.7349709       0      yes
42     42 response 2.2987358 0.12851060 2.0601684 2.5649293       0      yes
43     43 response 0.6992588 0.06369030 0.5849359 0.8359255       0       no
44     44 response 0.9571244 0.07498949 0.8208765 1.1159865       0      yes
45     45 response 1.2054476 0.07523205 1.0666570 1.3622972       0      yes
46     46 response 0.9494681 0.08362332 0.7989356 1.1283634       0       no
47     47 response 1.2054476 0.07523205 1.0666570 1.3622972       0      yes
48     48 response 2.2299485 0.12427095 1.9992124 2.4873147       0      yes
49     49 response 0.8141029 0.07692276 0.6764741 0.9797323       0      yes
50     50 response 2.2987358 0.12851060 2.0601684 2.5649293       0      yes
51     51 response 0.7173943 0.06475348 0.6010723 0.8562274       0       no
52     52 response 1.2426320 0.07826874 1.0983191 1.4059068       0      yes
53     53 response 0.9866488 0.07787951 0.8452290 1.1517303       0      yes
54     54 response 1.5650294 0.08231414 1.4117337 1.7349709       0      yes
55     55 response 0.9494681 0.08362332 0.7989356 1.1283634       0       no
56     56 response 2.2299485 0.12427095 1.9992124 2.4873147       0      yes
57     57 response 0.8141029 0.07692276 0.6764741 0.9797323       0      yes
58     58 response 2.2299485 0.12427095 1.9992124 2.4873147       0      yes
59     59 response 0.7173943 0.06475348 0.6010723 0.8562274       0       no
60     60 response 1.5650294 0.08231414 1.4117337 1.7349709       0      yes
61     61 response 2.2299485 0.12427095 1.9992124 2.4873147       0      yes
62     62 response 1.5650294 0.08231414 1.4117337 1.7349709       0      yes
63     63 response 0.7173943 0.06475348 0.6010723 0.8562274       0       no
64     64 response 0.9494681 0.08362332 0.7989356 1.1283634       0       no
65     65 response 1.1958049 0.10787230 1.0020153 1.4270735       0       no
66     66 response 2.2987358 0.12851060 2.0601684 2.5649293       0      yes
67     67 response 1.5650294 0.08231414 1.4117337 1.7349709       0      yes
68     68 response 2.2299485 0.12427095 1.9992124 2.4873147       0      yes
69     69 response 0.7599813 0.06737756 0.6387605 0.9042068       0       no
70     70 response 0.7599813 0.06737756 0.6387605 0.9042068       0       no
71     71 response 0.9210563 0.07913696 0.7783073 1.0899868       0       no
72     72 response 1.5181975 0.07908880 1.3708372 1.6813985       0      yes
73     73 response 0.6837339 0.06280417 0.5710843 0.8186042       0       no
74     74 response 2.2987358 0.12851060 2.0601684 2.5649293       0      yes
75     75 response 2.2987358 0.12851060 2.0601684 2.5649293       0      yes
76     76 response 1.2426320 0.07826874 1.0983191 1.4059068       0      yes
77     77 response 0.7897418 0.07416668 0.6569716 0.9493440       0      yes
78     78 response 0.9866488 0.07787951 0.8452290 1.1517303       0      yes
79     79 response 0.7599813 0.06737756 0.6387605 0.9042068       0       no
80     80 response 2.2987358 0.12851060 2.0601684 2.5649293       0      yes
81     81 response 0.8141029 0.07692276 0.6764741 0.9797323       0      yes
82     82 response 2.2987358 0.12851060 2.0601684 2.5649293       0      yes
83     83 response 0.7173943 0.06475348 0.6010723 0.8562274       0       no
84     84 response 0.9494681 0.08362332 0.7989356 1.1283634       0       no
85     85 response 0.7897418 0.07416668 0.6569716 0.9493440       0      yes
86     86 response 0.6992588 0.06369030 0.5849359 0.8359255       0       no
87     87 response 2.2299485 0.12427095 1.9992124 2.4873147       0      yes
88     88 response 0.7599813 0.06737756 0.6387605 0.9042068       0       no
89     89 response 0.7897418 0.07416668 0.6569716 0.9493440       0      yes
90     90 response 2.2299485 0.12427095 1.9992124 2.4873147       0      yes
91     91 response 2.2299485 0.12427095 1.9992124 2.4873147       0      yes
92     92 response 1.5650294 0.08231414 1.4117337 1.7349709       0      yes
93     93 response 1.5650294 0.08231414 1.4117337 1.7349709       0      yes
94     94 response 2.2987358 0.12851060 2.0601684 2.5649293       0      yes
95     95 response 0.7208288 0.06721263 0.6004310 0.8653687       0       no
96     96 response 2.2299485 0.12427095 1.9992124 2.4873147       0      yes
97     97 response 2.2987358 0.12851060 2.0601684 2.5649293       0      yes
98     98 response 1.5650294 0.08231414 1.4117337 1.7349709       0      yes
99     99 response 2.2987358 0.12851060 2.0601684 2.5649293       0      yes
100   100 response 2.2987358 0.12851060 2.0601684 2.5649293       0      yes
101   101 response 1.5181975 0.07908880 1.3708372 1.6813985       0      yes
102   102 response 1.5650294 0.08231414 1.4117337 1.7349709       0      yes
103   103 response 0.6837339 0.06280417 0.5710843 0.8186042       0       no
104   104 response 2.2987358 0.12851060 2.0601684 2.5649293       0      yes
105   105 response 2.2299485 0.12427095 1.9992124 2.4873147       0      yes
106   106 response 0.9866488 0.07787951 0.8452290 1.1517303       0      yes
107   107 response 1.2426320 0.07826874 1.0983191 1.4059068       0      yes
108   108 response 2.2987358 0.12851060 2.0601684 2.5649293       0      yes
109   109 response 2.2987358 0.12851060 2.0601684 2.5649293       0      yes
110   110 response 0.7599813 0.06737756 0.6387605 0.9042068       0       no
111   111 response 1.2054476 0.07523205 1.0666570 1.3622972       0      yes
112   112 response 2.2299485 0.12427095 1.9992124 2.4873147       0      yes
113   113 response 0.7599813 0.06737756 0.6387605 0.9042068       0       no
114   114 response 1.5650294 0.08231414 1.4117337 1.7349709       0      yes
115   115 response 1.1600217 0.10233572 0.9758300 1.3789804       0       no
116   116 response 2.2121107 0.26495535 1.7492596 2.7974313       0       no
117   117 response 2.2987358 0.12851060 2.0601684 2.5649293       0      yes
118   118 response 1.5060531 0.14741307 1.2431523 1.8245519       0       no
119   119 response 0.7173943 0.06475348 0.6010723 0.8562274       0       no
120   120 response 1.2054476 0.07523205 1.0666570 1.3622972       0      yes
121   121 response 1.2054476 0.07523205 1.0666570 1.3622972       0      yes
122   122 response 2.2299485 0.12427095 1.9992124 2.4873147       0      yes
123   123 response 2.2987358 0.12851060 2.0601684 2.5649293       0      yes
124   124 response 1.2054476 0.07523205 1.0666570 1.3622972       0      yes
125   125 response 0.7599813 0.06737756 0.6387605 0.9042068       0       no
126   126 response 2.2987358 0.12851060 2.0601684 2.5649293       0      yes
127   127 response 0.6837339 0.06280417 0.5710843 0.8186042       0       no
128   128 response 0.7834244 0.07116344 0.6556575 0.9360891       0       no
129   129 response 0.7834244 0.07116344 0.6556575 0.9360891       0       no
130   130 response 0.7834244 0.07116344 0.6556575 0.9360891       0       no
131   131 response 1.5181975 0.07908880 1.3708372 1.6813985       0      yes
132   132 response 2.2987358 0.12851060 2.0601684 2.5649293       0      yes
133   133 response 0.7599813 0.06737756 0.6387605 0.9042068       0       no
134   134 response 0.9494681 0.08362332 0.7989356 1.1283634       0       no
135   135 response 1.5181975 0.07908880 1.3708372 1.6813985       0      yes
136   136 response 2.2299485 0.12427095 1.9992124 2.4873147       0      yes
137   137 response 0.7173943 0.06475348 0.6010723 0.8562274       0       no
138   138 response 2.2299485 0.12427095 1.9992124 2.4873147       0      yes
139   139 response 0.9866488 0.07787951 0.8452290 1.1517303       0      yes
140   140 response 0.9494681 0.08362332 0.7989356 1.1283634       0       no
141   141 response 1.5181975 0.07908880 1.3708372 1.6813985       0      yes
142   142 response 2.2299485 0.12427095 1.9992124 2.4873147       0      yes
143   143 response 1.2426320 0.07826874 1.0983191 1.4059068       0      yes
144   144 response 2.2987358 0.12851060 2.0601684 2.5649293       0      yes
145   145 response 0.9494681 0.08362332 0.7989356 1.1283634       0       no
146   146 response 0.9866488 0.07787951 0.8452290 1.1517303       0      yes
147   147 response 0.9866488 0.07787951 0.8452290 1.1517303       0      yes
148   148 response 2.2299485 0.12427095 1.9992124 2.4873147       0      yes
149   149 response 2.2299485 0.12427095 1.9992124 2.4873147       0      yes
150   150 response 1.2426320 0.07826874 1.0983191 1.4059068       0      yes
151   151 response 1.2426320 0.07826874 1.0983191 1.4059068       0      yes
152   152 response 0.7395237 0.06835445 0.6169859 0.8863984       0       no
153   153 response 0.9866488 0.07787951 0.8452290 1.1517303       0      yes
154   154 response 2.2987358 0.12851060 2.0601684 2.5649293       0      yes
155   155 response 0.7684831 0.07638978 0.6324440 0.9337844       0      yes
156   156 response 2.2987358 0.12851060 2.0601684 2.5649293       0      yes
157   157 response 2.2299485 0.12427095 1.9992124 2.4873147       0      yes
158   158 response 2.2299485 0.12427095 1.9992124 2.4873147       0      yes
159   159 response 1.2426320 0.07826874 1.0983191 1.4059068       0      yes
160   160 response 0.9866488 0.07787951 0.8452290 1.1517303       0      yes
161   161 response 0.9571244 0.07498949 0.8208765 1.1159865       0      yes
162   162 response 0.9571244 0.07498949 0.8208765 1.1159865       0      yes
163   163 response 2.2987358 0.12851060 2.0601684 2.5649293       0      yes
164   164 response 2.2299485 0.12427095 1.9992124 2.4873147       0      yes
165   165 response 1.5181975 0.07908880 1.3708372 1.6813985       0      yes
166   166 response 0.7897418 0.07416668 0.6569716 0.9493440       0      yes
167   167 response 2.2299485 0.12427095 1.9992124 2.4873147       0      yes
168   168 response 0.7599813 0.06737756 0.6387605 0.9042068       0       no
169   169 response 2.2987358 0.12851060 2.0601684 2.5649293       0      yes
170   170 response 2.2987358 0.12851060 2.0601684 2.5649293       0      yes
171   171 response 2.2987358 0.12851060 2.0601684 2.5649293       0      yes
172   172 response 0.6837339 0.06280417 0.5710843 0.8186042       0       no
173   173 response 1.5181975 0.07908880 1.3708372 1.6813985       0      yes
174   174 response 2.2299485 0.12427095 1.9992124 2.4873147       0      yes
175   175 response 0.7599813 0.06737756 0.6387605 0.9042068       0       no
176   176 response 0.7324256 0.07585460 0.5978713 0.8972620       0      yes
177   177 response 0.9210563 0.07913696 0.7783073 1.0899868       0       no
178   178 response 1.5181975 0.07908880 1.3708372 1.6813985       0      yes
179   179 response 1.2054476 0.07523205 1.0666570 1.3622972       0      yes
180   180 response 2.2299485 0.12427095 1.9992124 2.4873147       0      yes
181   181 response 2.2299485 0.12427095 1.9992124 2.4873147       0      yes
182   182 response 0.8141029 0.07692276 0.6764741 0.9797323       0      yes
183   183 response 0.9571244 0.07498949 0.8208765 1.1159865       0      yes
184   184 response 1.1600217 0.10233572 0.9758300 1.3789804       0       no
185   185 response 0.7266414 0.07342315 0.5960887 0.8857872       0      yes
186   186 response 1.2054476 0.07523205 1.0666570 1.3622972       0      yes
187   187 response 0.7395237 0.06835445 0.6169859 0.8863984       0       no
188   188 response 0.9866488 0.07787951 0.8452290 1.1517303       0      yes
189   189 response 1.5650294 0.08231414 1.4117337 1.7349709       0      yes
190   190 response 2.2987358 0.12851060 2.0601684 2.5649293       0      yes
191   191 response 0.7395237 0.06835445 0.6169859 0.8863984       0       no
192   192 response 1.2054476 0.07523205 1.0666570 1.3622972       0      yes
193   193 response 0.7490561 0.07611485 0.6137904 0.9141314       0      yes
194   194 response 2.2987358 0.12851060 2.0601684 2.5649293       0      yes
195   195 response 2.2299485 0.12427095 1.9992124 2.4873147       0      yes
196   196 response 0.9866488 0.07787951 0.8452290 1.1517303       0      yes
197   197 response 2.2987358 0.12851060 2.0601684 2.5649293       0      yes
198   198 response 0.7599813 0.06737756 0.6387605 0.9042068       0       no
199   199 response 2.2987358 0.12851060 2.0601684 2.5649293       0      yes
200   200 response 0.7599813 0.06737756 0.6387605 0.9042068       0       no
201   201 response 0.9866488 0.07787951 0.8452290 1.1517303       0      yes
202   202 response 0.9866488 0.07787951 0.8452290 1.1517303       0      yes
203   203 response 0.7599813 0.06737756 0.6387605 0.9042068       0       no
204   204 response 0.7395237 0.06835445 0.6169859 0.8863984       0       no
205   205 response 1.2054476 0.07523205 1.0666570 1.3622972       0      yes
206   206 response 0.7173943 0.06475348 0.6010723 0.8562274       0       no
207   207 response 2.2299485 0.12427095 1.9992124 2.4873147       0      yes
208   208 response 0.7599813 0.06737756 0.6387605 0.9042068       0       no
209   209 response 1.4609860 0.14047645 1.2100449 1.7639677       0       no
210   210 response 2.2299485 0.12427095 1.9992124 2.4873147       0      yes
211   211 response 2.2299485 0.12427095 1.9992124 2.4873147       0      yes
212   212 response 0.9494681 0.08362332 0.7989356 1.1283634       0       no
213   213 response 1.2426320 0.07826874 1.0983191 1.4059068       0      yes
214   214 response 0.7395237 0.06835445 0.6169859 0.8863984       0       no
215   215 response 0.9210563 0.07913696 0.7783073 1.0899868       0       no
216   216 response 0.7834244 0.07116344 0.6556575 0.9360891       0       no
217   217 response 0.7834244 0.07116344 0.6556575 0.9360891       0       no
218   218 response 0.9494681 0.08362332 0.7989356 1.1283634       0       no
219   219 response 0.9571244 0.07498949 0.8208765 1.1159865       0      yes
220   220 response 2.2987358 0.12851060 2.0601684 2.5649293       0      yes
221   221 response 1.5181975 0.07908880 1.3708372 1.6813985       0      yes
222   222 response 2.2987358 0.12851060 2.0601684 2.5649293       0      yes
223   223 response 2.2299485 0.12427095 1.9992124 2.4873147       0      yes
224   224 response 0.7834244 0.07116344 0.6556575 0.9360891       0       no
225   225 response 1.5181975 0.07908880 1.3708372 1.6813985       0      yes
226   226 response 1.2054476 0.07523205 1.0666570 1.3622972       0      yes
227   227 response 1.5181975 0.07908880 1.3708372 1.6813985       0      yes
228   228 response 0.8141029 0.07692276 0.6764741 0.9797323       0      yes
229   229 response 2.2987358 0.12851060 2.0601684 2.5649293       0      yes
230   230 response 0.9866488 0.07787951 0.8452290 1.1517303       0      yes
231   231 response 2.2299485 0.12427095 1.9992124 2.4873147       0      yes
232   232 response 1.2054476 0.07523205 1.0666570 1.3622972       0      yes
233   233 response 2.2299485 0.12427095 1.9992124 2.4873147       0      yes
234   234 response 0.9494681 0.08362332 0.7989356 1.1283634       0       no
235   235 response 0.7395237 0.06835445 0.6169859 0.8863984       0       no
236   236 response 0.9866488 0.07787951 0.8452290 1.1517303       0      yes
237   237 response 0.7173943 0.06475348 0.6010723 0.8562274       0       no
238   238 response 2.2299485 0.12427095 1.9992124 2.4873147       0      yes
239   239 response 1.5650294 0.08231414 1.4117337 1.7349709       0      yes
240   240 response 2.2299485 0.12427095 1.9992124 2.4873147       0      yes
241   241 response 1.2426320 0.07826874 1.0983191 1.4059068       0      yes
242   242 response 2.2299485 0.12427095 1.9992124 2.4873147       0      yes
243   243 response 2.2987358 0.12851060 2.0601684 2.5649293       0      yes
244   244 response 0.9866488 0.07787951 0.8452290 1.1517303       0      yes
245   245 response 2.2299485 0.12427095 1.9992124 2.4873147       0      yes
246   246 response 0.7684831 0.07638978 0.6324440 0.9337844       0      yes
247   247 response 0.7599813 0.06737756 0.6387605 0.9042068       0       no
248   248 response 2.2299485 0.12427095 1.9992124 2.4873147       0      yes
249   249 response 2.2299485 0.12427095 1.9992124 2.4873147       0      yes
250   250 response 2.2987358 0.12851060 2.0601684 2.5649293       0      yes
251   251 response 2.2299485 0.12427095 1.9992124 2.4873147       0      yes
252   252 response 2.2987358 0.12851060 2.0601684 2.5649293       0      yes
253   253 response 1.2426320 0.07826874 1.0983191 1.4059068       0      yes
254   254 response 1.2054476 0.07523205 1.0666570 1.3622972       0      yes
255   255 response 0.7834244 0.07116344 0.6556575 0.9360891       0       no
256   256 response 0.9571244 0.07498949 0.8208765 1.1159865       0      yes
257   257 response 0.7599813 0.06737756 0.6387605 0.9042068       0       no
258   258 response 2.2987358 0.12851060 2.0601684 2.5649293       0      yes
259   259 response 2.2987358 0.12851060 2.0601684 2.5649293       0      yes
260   260 response 1.2054476 0.07523205 1.0666570 1.3622972       0      yes
261   261 response 0.9571244 0.07498949 0.8208765 1.1159865       0      yes
262   262 response 2.2987358 0.12851060 2.0601684 2.5649293       0      yes
263   263 response 1.2426320 0.07826874 1.0983191 1.4059068       0      yes
264   264 response 0.7599813 0.06737756 0.6387605 0.9042068       0       no
265   265 response 0.7897418 0.07416668 0.6569716 0.9493440       0      yes
266   266 response 2.2987358 0.12851060 2.0601684 2.5649293       0      yes
267   267 response 2.2299485 0.12427095 1.9992124 2.4873147       0      yes
268   268 response 1.5181975 0.07908880 1.3708372 1.6813985       0      yes
269   269 response 2.2987358 0.12851060 2.0601684 2.5649293       0      yes
270   270 response 0.9571244 0.07498949 0.8208765 1.1159865       0      yes
271   271 response 2.2987358 0.12851060 2.0601684 2.5649293       0      yes
272   272 response 0.7834244 0.07116344 0.6556575 0.9360891       0       no
273   273 response 0.6837339 0.06280417 0.5710843 0.8186042       0       no
274   274 response 1.5650294 0.08231414 1.4117337 1.7349709       0      yes
275   275 response 0.9210563 0.07913696 0.7783073 1.0899868       0       no
276   276 response 1.2054476 0.07523205 1.0666570 1.3622972       0      yes
277   277 response 0.9866488 0.07787951 0.8452290 1.1517303       0      yes
278   278 response 2.2299485 0.12427095 1.9992124 2.4873147       0      yes
279   279 response 2.2121107 0.26495535 1.7492596 2.7974313       0       no
280   280 response 0.7834244 0.07116344 0.6556575 0.9360891       0       no
281   281 response 2.2987358 0.12851060 2.0601684 2.5649293       0      yes
282   282 response 2.2299485 0.12427095 1.9992124 2.4873147       0      yes
283   283 response 1.2054476 0.07523205 1.0666570 1.3622972       0      yes
284   284 response 2.2299485 0.12427095 1.9992124 2.4873147       0      yes
285   285 response 1.2054476 0.07523205 1.0666570 1.3622972       0      yes
286   286 response 0.7834244 0.07116344 0.6556575 0.9360891       0       no
287   287 response 0.7834244 0.07116344 0.6556575 0.9360891       0       no
288   288 response 0.9866488 0.07787951 0.8452290 1.1517303       0      yes
289   289 response 1.2054476 0.07523205 1.0666570 1.3622972       0      yes
290   290 response 2.2299485 0.12427095 1.9992124 2.4873147       0      yes
291   291 response 1.2054476 0.07523205 1.0666570 1.3622972       0      yes
292   292 response 0.7834244 0.07116344 0.6556575 0.9360891       0       no
293   293 response 2.2987358 0.12851060 2.0601684 2.5649293       0      yes
294   294 response 0.7599813 0.06737756 0.6387605 0.9042068       0       no
295   295 response 1.5181975 0.07908880 1.3708372 1.6813985       0      yes
296   296 response 0.9494681 0.08362332 0.7989356 1.1283634       0       no
297   297 response 0.6992588 0.06369030 0.5849359 0.8359255       0       no
298   298 response 0.9210563 0.07913696 0.7783073 1.0899868       0       no
299   299 response 2.2987358 0.12851060 2.0601684 2.5649293       0      yes
300   300 response 1.5650294 0.08231414 1.4117337 1.7349709       0      yes
301   301 response 1.2054476 0.07523205 1.0666570 1.3622972       0      yes
302   302 response 0.9866488 0.07787951 0.8452290 1.1517303       0      yes
303   303 response 0.9866488 0.07787951 0.8452290 1.1517303       0      yes
304   304 response 0.7834244 0.07116344 0.6556575 0.9360891       0       no
305   305 response 0.9571244 0.07498949 0.8208765 1.1159865       0      yes
306   306 response 0.7173943 0.06475348 0.6010723 0.8562274       0       no
307   307 response 1.5650294 0.08231414 1.4117337 1.7349709       0      yes
308   308 response 0.7173943 0.06475348 0.6010723 0.8562274       0       no
309   309 response 2.2987358 0.12851060 2.0601684 2.5649293       0      yes
310   310 response 0.9494681 0.08362332 0.7989356 1.1283634       0       no
311   311 response 0.7599813 0.06737756 0.6387605 0.9042068       0       no
312   312 response 1.2054476 0.07523205 1.0666570 1.3622972       0      yes
313   313 response 2.2987358 0.12851060 2.0601684 2.5649293       0      yes
314   314 response 0.9866488 0.07787951 0.8452290 1.1517303       0      yes
315   315 response 2.2299485 0.12427095 1.9992124 2.4873147       0      yes
316   316 response 0.9866488 0.07787951 0.8452290 1.1517303       0      yes
317   317 response 0.9571244 0.07498949 0.8208765 1.1159865       0      yes
318   318 response 2.2987358 0.12851060 2.0601684 2.5649293       0      yes
319   319 response 0.7599813 0.06737756 0.6387605 0.9042068       0       no
320   320 response 2.2987358 0.12851060 2.0601684 2.5649293       0      yes
321   321 response 0.6992588 0.06369030 0.5849359 0.8359255       0       no
322   322 response 0.7599813 0.06737756 0.6387605 0.9042068       0       no
323   323 response 0.9866488 0.07787951 0.8452290 1.1517303       0      yes
324   324 response 2.2299485 0.12427095 1.9992124 2.4873147       0      yes
325   325 response 0.7599813 0.06737756 0.6387605 0.9042068       0       no
326   326 response 0.9494681 0.08362332 0.7989356 1.1283634       0       no
327   327 response 2.2987358 0.12851060 2.0601684 2.5649293       0      yes
328   328 response 1.4609860 0.14047645 1.2100449 1.7639677       0       no
329   329 response 1.5181975 0.07908880 1.3708372 1.6813985       0      yes
330   330 response 0.8141029 0.07692276 0.6764741 0.9797323       0      yes
331   331 response 0.7834244 0.07116344 0.6556575 0.9360891       0       no
332   332 response 1.5181975 0.07908880 1.3708372 1.6813985       0      yes
333   333 response 2.2299485 0.12427095 1.9992124 2.4873147       0      yes
334   334 response 1.5181975 0.07908880 1.3708372 1.6813985       0      yes
335   335 response 0.9866488 0.07787951 0.8452290 1.1517303       0      yes
336   336 response 2.2299485 0.12427095 1.9992124 2.4873147       0      yes
337   337 response 0.7173943 0.06475348 0.6010723 0.8562274       0       no
338   338 response 2.2299485 0.12427095 1.9992124 2.4873147       0      yes
339   339 response 2.2299485 0.12427095 1.9992124 2.4873147       0      yes
340   340 response 1.5181975 0.07908880 1.3708372 1.6813985       0      yes
341   341 response 1.1958049 0.10787230 1.0020153 1.4270735       0       no
342   342 response 2.2299485 0.12427095 1.9992124 2.4873147       0      yes
343   343 response 0.8141029 0.07692276 0.6764741 0.9797323       0      yes
344   344 response 0.7454871 0.07367786 0.6142063 0.9048278       0      yes
345   345 response 0.9866488 0.07787951 0.8452290 1.1517303       0      yes
346   346 response 0.7048250 0.06625926 0.5862211 0.8474247       0       no
347   347 response 0.8141029 0.07692276 0.6764741 0.9797323       0      yes
348   348 response 1.2426320 0.07826874 1.0983191 1.4059068       0      yes
349   349 response 2.2299485 0.12427095 1.9992124 2.4873147       0      yes
350   350 response 0.7834244 0.07116344 0.6556575 0.9360891       0       no
351   351 response 0.7599813 0.06737756 0.6387605 0.9042068       0       no
352   352 response 0.7599813 0.06737756 0.6387605 0.9042068       0       no
353   353 response 0.7048250 0.06625926 0.5862211 0.8474247       0       no
354   354 response 0.9571244 0.07498949 0.8208765 1.1159865       0      yes
355   355 response 0.9571244 0.07498949 0.8208765 1.1159865       0      yes
356   356 response 2.2299485 0.12427095 1.9992124 2.4873147       0      yes
357   357 response 2.2299485 0.12427095 1.9992124 2.4873147       0      yes
358   358 response 1.2426320 0.07826874 1.0983191 1.4059068       0      yes
359   359 response 0.7395237 0.06835445 0.6169859 0.8863984       0       no
360   360 response 0.7048250 0.06625926 0.5862211 0.8474247       0       no
361   361 response 1.5650294 0.08231414 1.4117337 1.7349709       0      yes
362   362 response 0.6992588 0.06369030 0.5849359 0.8359255       0       no
363   363 response 2.2299485 0.12427095 1.9992124 2.4873147       0      yes
364   364 response 1.5181975 0.07908880 1.3708372 1.6813985       0      yes
365   365 response 2.2987358 0.12851060 2.0601684 2.5649293       0      yes
366   366 response 0.9866488 0.07787951 0.8452290 1.1517303       0      yes
367   367 response 1.2054476 0.07523205 1.0666570 1.3622972       0      yes
368   368 response 1.5181975 0.07908880 1.3708372 1.6813985       0      yes
369   369 response 1.5181975 0.07908880 1.3708372 1.6813985       0      yes
370   370 response 0.9210563 0.07913696 0.7783073 1.0899868       0       no
371   371 response 1.2054476 0.07523205 1.0666570 1.3622972       0      yes
372   372 response 2.2987358 0.12851060 2.0601684 2.5649293       0      yes
373   373 response 1.2426320 0.07826874 1.0983191 1.4059068       0      yes
374   374 response 0.7599813 0.06737756 0.6387605 0.9042068       0       no
375   375 response 0.7834244 0.07116344 0.6556575 0.9360891       0       no
376   376 response 0.7897418 0.07416668 0.6569716 0.9493440       0      yes
377   377 response 1.2054476 0.07523205 1.0666570 1.3622972       0      yes
378   378 response 2.2299485 0.12427095 1.9992124 2.4873147       0      yes
379   379 response 0.7173943 0.06475348 0.6010723 0.8562274       0       no
380   380 response 0.7897418 0.07416668 0.6569716 0.9493440       0      yes
381   381 response 0.9571244 0.07498949 0.8208765 1.1159865       0      yes
382   382 response 2.2987358 0.12851060 2.0601684 2.5649293       0      yes
383   383 response 1.5650294 0.08231414 1.4117337 1.7349709       0      yes
384   384 response 2.2987358 0.12851060 2.0601684 2.5649293       0      yes
385   385 response 1.2054476 0.07523205 1.0666570 1.3622972       0      yes
386   386 response 0.7599813 0.06737756 0.6387605 0.9042068       0       no
387   387 response 1.5181975 0.07908880 1.3708372 1.6813985       0      yes
388   388 response 0.7173943 0.06475348 0.6010723 0.8562274       0       no
389   389 response 0.7599813 0.06737756 0.6387605 0.9042068       0       no
390   390 response 2.2299485 0.12427095 1.9992124 2.4873147       0      yes
391   391 response 1.2054476 0.07523205 1.0666570 1.3622972       0      yes
392   392 response 2.2987358 0.12851060 2.0601684 2.5649293       0      yes
393   393 response 2.2299485 0.12427095 1.9992124 2.4873147       0      yes
394   394 response 2.2987358 0.12851060 2.0601684 2.5649293       0      yes
395   395 response 1.2054476 0.07523205 1.0666570 1.3622972       0      yes
396   396 response 0.9494681 0.08362332 0.7989356 1.1283634       0       no
397   397 response 0.7599813 0.06737756 0.6387605 0.9042068       0       no
398   398 response 0.7173943 0.06475348 0.6010723 0.8562274       0       no
399   399 response 2.2299485 0.12427095 1.9992124 2.4873147       0      yes
400   400 response 0.7834244 0.07116344 0.6556575 0.9360891       0       no
401   401 response 0.9866488 0.07787951 0.8452290 1.1517303       0      yes
402   402 response 0.9571244 0.07498949 0.8208765 1.1159865       0      yes
403   403 response 2.2987358 0.12851060 2.0601684 2.5649293       0      yes
404   404 response 0.7834244 0.07116344 0.6556575 0.9360891       0       no
405   405 response 2.2121107 0.26495535 1.7492596 2.7974313       0       no
406   406 response 1.1958049 0.10787230 1.0020153 1.4270735       0       no
407   407 response 2.2987358 0.12851060 2.0601684 2.5649293       0      yes
408   408 response 2.2987358 0.12851060 2.0601684 2.5649293       0      yes
409   409 response 1.1600217 0.10233572 0.9758300 1.3789804       0       no
410   410 response 1.2054476 0.07523205 1.0666570 1.3622972       0      yes
411   411 response 0.9210563 0.07913696 0.7783073 1.0899868       0       no
412   412 response 1.2426320 0.07826874 1.0983191 1.4059068       0      yes
413   413 response 1.1958049 0.10787230 1.0020153 1.4270735       0       no
414   414 response 1.5650294 0.08231414 1.4117337 1.7349709       0      yes
415   415 response 0.7599813 0.06737756 0.6387605 0.9042068       0       no
416   416 response 0.9571244 0.07498949 0.8208765 1.1159865       0      yes
417   417 response 2.1459156 0.25430191 1.7011434 2.7069756       0       no
418   418 response 2.2987358 0.12851060 2.0601684 2.5649293       0      yes
419   419 response 1.2054476 0.07523205 1.0666570 1.3622972       0      yes
420   420 response 1.4609860 0.14047645 1.2100449 1.7639677       0       no
421   421 response 0.9866488 0.07787951 0.8452290 1.1517303       0      yes
422   422 response 0.9571244 0.07498949 0.8208765 1.1159865       0      yes
423   423 response 2.2299485 0.12427095 1.9992124 2.4873147       0      yes
424   424 response 0.7599813 0.06737756 0.6387605 0.9042068       0       no
425   425 response 1.2426320 0.07826874 1.0983191 1.4059068       0      yes
426   426 response 0.9866488 0.07787951 0.8452290 1.1517303       0      yes
427   427 response 0.7048250 0.06625926 0.5862211 0.8474247       0       no
428   428 response 1.2054476 0.07523205 1.0666570 1.3622972       0      yes
429   429 response 2.2299485 0.12427095 1.9992124 2.4873147       0      yes
430   430 response 1.5650294 0.08231414 1.4117337 1.7349709       0      yes
431   431 response 2.2299485 0.12427095 1.9992124 2.4873147       0      yes
432   432 response 0.7599813 0.06737756 0.6387605 0.9042068       0       no
433   433 response 0.9866488 0.07787951 0.8452290 1.1517303       0      yes
434   434 response 2.2299485 0.12427095 1.9992124 2.4873147       0      yes
435   435 response 1.2054476 0.07523205 1.0666570 1.3622972       0      yes
436   436 response 1.2054476 0.07523205 1.0666570 1.3622972       0      yes
437   437 response 2.2299485 0.12427095 1.9992124 2.4873147       0      yes
438   438 response 2.2299485 0.12427095 1.9992124 2.4873147       0      yes
439   439 response 1.2426320 0.07826874 1.0983191 1.4059068       0      yes
440   440 response 1.2426320 0.07826874 1.0983191 1.4059068       0      yes
441   441 response 2.2299485 0.12427095 1.9992124 2.4873147       0      yes
442   442 response 0.9866488 0.07787951 0.8452290 1.1517303       0      yes
443   443 response 1.2054476 0.07523205 1.0666570 1.3622972       0      yes
444   444 response 0.7599813 0.06737756 0.6387605 0.9042068       0       no
445   445 response 0.9571244 0.07498949 0.8208765 1.1159865       0      yes
446   446 response 0.9210563 0.07913696 0.7783073 1.0899868       0       no
447   447 response 0.7599813 0.06737756 0.6387605 0.9042068       0       no
448   448 response 2.2121107 0.26495535 1.7492596 2.7974313       0       no
449   449 response 1.5650294 0.08231414 1.4117337 1.7349709       0      yes
450   450 response 2.2987358 0.12851060 2.0601684 2.5649293       0      yes
451   451 response 0.9571244 0.07498949 0.8208765 1.1159865       0      yes
452   452 response 0.7834244 0.07116344 0.6556575 0.9360891       3       no
453   453 response 0.9571244 0.07498949 0.8208765 1.1159865       3      yes
454   454 response 2.2987358 0.12851060 2.0601684 2.5649293       7      yes
455   455 response 1.5181975 0.07908880 1.3708372 1.6813985      12      yes
456   456 response 0.7048250 0.06625926 0.5862211 0.8474247       1       no
457   457 response 0.7897418 0.07416668 0.6569716 0.9493440       1      yes
458   458 response 2.2987358 0.12851060 2.0601684 2.5649293      12      yes
459   459 response 0.7599813 0.06737756 0.6387605 0.9042068       7       no
460   460 response 2.2987358 0.12851060 2.0601684 2.5649293       2      yes
461   461 response 2.2299485 0.12427095 1.9992124 2.4873147       3      yes
462   462 response 2.2299485 0.12427095 1.9992124 2.4873147       1      yes
463   463 response 2.2299485 0.12427095 1.9992124 2.4873147       7      yes
464   464 response 2.2299485 0.12427095 1.9992124 2.4873147      12      yes
465   465 response 1.5650294 0.08231414 1.4117337 1.7349709      12      yes
466   466 response 2.2299485 0.12427095 1.9992124 2.4873147      12      yes
467   467 response 0.9494681 0.08362332 0.7989356 1.1283634       3       no
468   468 response 1.5650294 0.08231414 1.4117337 1.7349709       7      yes
469   469 response 0.9210563 0.07913696 0.7783073 1.0899868       7       no
470   470 response 2.2987358 0.12851060 2.0601684 2.5649293       1      yes
471   471 response 2.2299485 0.12427095 1.9992124 2.4873147       1      yes
472   472 response 0.9571244 0.07498949 0.8208765 1.1159865       7      yes
473   473 response 1.2054476 0.07523205 1.0666570 1.3622972       1      yes
474   474 response 0.8141029 0.07692276 0.6764741 0.9797323      12      yes
475   475 response 1.2054476 0.07523205 1.0666570 1.3622972      12      yes
476   476 response 2.2299485 0.12427095 1.9992124 2.4873147       3      yes
477   477 response 1.5181975 0.07908880 1.3708372 1.6813985       7      yes
478   478 response 0.7834244 0.07116344 0.6556575 0.9360891       1       no
479   479 response 0.9494681 0.08362332 0.7989356 1.1283634       1       no
480   480 response 1.2054476 0.07523205 1.0666570 1.3622972       1      yes
481   481 response 1.5181975 0.07908880 1.3708372 1.6813985       3      yes
482   482 response 0.9866488 0.07787951 0.8452290 1.1517303       3      yes
483   483 response 0.7173943 0.06475348 0.6010723 0.8562274       1       no
484   484 response 1.5181975 0.07908880 1.3708372 1.6813985       1      yes
485   485 response 1.2054476 0.07523205 1.0666570 1.3622972       7      yes
486   486 response 2.2987358 0.12851060 2.0601684 2.5649293       7      yes
487   487 response 1.4609860 0.14047645 1.2100449 1.7639677       7       no
488   488 response 1.5181975 0.07908880 1.3708372 1.6813985      12      yes
489   489 response 2.2987358 0.12851060 2.0601684 2.5649293       7      yes
490   490 response 2.2299485 0.12427095 1.9992124 2.4873147       7      yes
491   491 response 2.2987358 0.12851060 2.0601684 2.5649293       1      yes
492   492 response 2.2987358 0.12851060 2.0601684 2.5649293       2      yes
493   493 response 0.9210563 0.07913696 0.7783073 1.0899868      12       no
494   494 response 1.2426320 0.07826874 1.0983191 1.4059068      12      yes
495   495 response 0.9866488 0.07787951 0.8452290 1.1517303       1      yes
496   496 response 2.2987358 0.12851060 2.0601684 2.5649293      12      yes
497   497 response 2.2299485 0.12427095 1.9992124 2.4873147      12      yes
498   498 response 0.9494681 0.08362332 0.7989356 1.1283634       7       no
499   499 response 1.2054476 0.07523205 1.0666570 1.3622972       7      yes
500   500 response 0.7490561 0.07611485 0.6137904 0.9141314       1      yes
501   501 response 2.2987358 0.12851060 2.0601684 2.5649293       3      yes
502   502 response 2.2987358 0.12851060 2.0601684 2.5649293      12      yes
503   503 response 0.9866488 0.07787951 0.8452290 1.1517303       7      yes
504   504 response 0.9494681 0.08362332 0.7989356 1.1283634       1       no
505   505 response 2.2299485 0.12427095 1.9992124 2.4873147       7      yes
506   506 response 0.9494681 0.08362332 0.7989356 1.1283634       1       no
507   507 response 1.5181975 0.07908880 1.3708372 1.6813985       1      yes
508   508 response 1.2426320 0.07826874 1.0983191 1.4059068       1      yes
509   509 response 1.2426320 0.07826874 1.0983191 1.4059068       7      yes
510   510 response 0.7834244 0.07116344 0.6556575 0.9360891       3       no
511   511 response 0.9866488 0.07787951 0.8452290 1.1517303       7      yes
512   512 response 2.2987358 0.12851060 2.0601684 2.5649293       7      yes
513   513 response 2.2299485 0.12427095 1.9992124 2.4873147       2      yes
514   514 response 0.9571244 0.07498949 0.8208765 1.1159865       7      yes
515   515 response 0.9866488 0.07787951 0.8452290 1.1517303       1      yes
516   516 response 1.2426320 0.07826874 1.0983191 1.4059068       7      yes
517   517 response 2.2987358 0.12851060 2.0601684 2.5649293       2      yes
518   518 response 2.2299485 0.12427095 1.9992124 2.4873147       7      yes
519   519 response 1.5650294 0.08231414 1.4117337 1.7349709       7      yes
520   520 response 2.2987358 0.12851060 2.0601684 2.5649293       3      yes
521   521 response 2.2299485 0.12427095 1.9992124 2.4873147       1      yes
522   522 response 1.2054476 0.07523205 1.0666570 1.3622972       2      yes
523   523 response 1.2426320 0.07826874 1.0983191 1.4059068      12      yes
524   524 response 0.9494681 0.08362332 0.7989356 1.1283634       1       no
525   525 response 1.2426320 0.07826874 1.0983191 1.4059068       3      yes
526   526 response 2.2299485 0.12427095 1.9992124 2.4873147      12      yes
527   527 response 2.2299485 0.12427095 1.9992124 2.4873147       7      yes
528   528 response 1.1600217 0.10233572 0.9758300 1.3789804       7       no
529   529 response 1.2054476 0.07523205 1.0666570 1.3622972       1      yes
530   530 response 0.8141029 0.07692276 0.6764741 0.9797323       1      yes
531   531 response 2.2299485 0.12427095 1.9992124 2.4873147      12      yes
532   532 response 1.5650294 0.08231414 1.4117337 1.7349709       7      yes
533   533 response 0.9866488 0.07787951 0.8452290 1.1517303       7      yes
534   534 response 0.9571244 0.07498949 0.8208765 1.1159865       1      yes
535   535 response 2.2299485 0.12427095 1.9992124 2.4873147      12      yes
536   536 response 1.5650294 0.08231414 1.4117337 1.7349709       1      yes
537   537 response 1.5650294 0.08231414 1.4117337 1.7349709      12      yes
538   538 response 1.2054476 0.07523205 1.0666570 1.3622972      12      yes
539   539 response 1.2054476 0.07523205 1.0666570 1.3622972       3      yes
540   540 response 1.5650294 0.08231414 1.4117337 1.7349709       3      yes
541   541 response 0.7454871 0.07367786 0.6142063 0.9048278      12      yes
542   542 response 2.2299485 0.12427095 1.9992124 2.4873147      12      yes
543   543 response 1.1600217 0.10233572 0.9758300 1.3789804       2       no
544   544 response 1.2426320 0.07826874 1.0983191 1.4059068       1      yes
545   545 response 0.9866488 0.07787951 0.8452290 1.1517303       7      yes
546   546 response 0.7897418 0.07416668 0.6569716 0.9493440       1      yes
547   547 response 2.1459156 0.25430191 1.7011434 2.7069756      12       no
548   548 response 2.2299485 0.12427095 1.9992124 2.4873147      12      yes
549   549 response 2.2987358 0.12851060 2.0601684 2.5649293       7      yes
550   550 response 1.5060531 0.14741307 1.2431523 1.8245519      12       no
551   551 response 2.2299485 0.12427095 1.9992124 2.4873147      12      yes
552   552 response 2.2987358 0.12851060 2.0601684 2.5649293       7      yes
553   553 response 2.2987358 0.12851060 2.0601684 2.5649293      12      yes
554   554 response 2.2299485 0.12427095 1.9992124 2.4873147       2      yes
555   555 response 2.2987358 0.12851060 2.0601684 2.5649293      12      yes
556   556 response 2.2987358 0.12851060 2.0601684 2.5649293      12      yes
557   557 response 1.5650294 0.08231414 1.4117337 1.7349709       7      yes
558   558 response 2.2987358 0.12851060 2.0601684 2.5649293       2      yes
559   559 response 2.2299485 0.12427095 1.9992124 2.4873147      12      yes
560   560 response 1.5650294 0.08231414 1.4117337 1.7349709       7      yes
561   561 response 2.2987358 0.12851060 2.0601684 2.5649293       2      yes
562   562 response 0.7599813 0.06737756 0.6387605 0.9042068       7       no
563   563 response 2.2299485 0.12427095 1.9992124 2.4873147       3      yes
564   564 response 2.2987358 0.12851060 2.0601684 2.5649293      12      yes
565   565 response 0.9210563 0.07913696 0.7783073 1.0899868      12       no
566   566 response 1.5181975 0.07908880 1.3708372 1.6813985       2      yes
567   567 response 0.9571244 0.07498949 0.8208765 1.1159865       1      yes
568   568 response 1.1958049 0.10787230 1.0020153 1.4270735      12       no
569   569 response 0.9571244 0.07498949 0.8208765 1.1159865       2      yes
570   570 response 2.2299485 0.12427095 1.9992124 2.4873147       7      yes
571   571 response 0.9210563 0.07913696 0.7783073 1.0899868       2       no
572   572 response 0.7454871 0.07367786 0.6142063 0.9048278      12      yes
573   573 response 2.2299485 0.12427095 1.9992124 2.4873147       7      yes
574   574 response 0.9210563 0.07913696 0.7783073 1.0899868       7       no
575   575 response 0.9866488 0.07787951 0.8452290 1.1517303       2      yes
576   576 response 0.7897418 0.07416668 0.6569716 0.9493440       1      yes
577   577 response 2.2299485 0.12427095 1.9992124 2.4873147       3      yes
578   578 response 1.2426320 0.07826874 1.0983191 1.4059068       1      yes
579   579 response 2.2121107 0.26495535 1.7492596 2.7974313      12       no
580   580 response 2.2987358 0.12851060 2.0601684 2.5649293       1      yes
581   581 response 0.9866488 0.07787951 0.8452290 1.1517303       1      yes
582   582 response 2.2987358 0.12851060 2.0601684 2.5649293       2      yes
583   583 response 2.2987358 0.12851060 2.0601684 2.5649293       7      yes
584   584 response 0.7834244 0.07116344 0.6556575 0.9360891       3       no
585   585 response 0.9866488 0.07787951 0.8452290 1.1517303       3      yes
586   586 response 2.2987358 0.12851060 2.0601684 2.5649293       2      yes
587   587 response 2.2299485 0.12427095 1.9992124 2.4873147      12      yes
588   588 response 2.2121107 0.26495535 1.7492596 2.7974313      12       no
589   589 response 2.2299485 0.12427095 1.9992124 2.4873147       3      yes
590   590 response 2.2299485 0.12427095 1.9992124 2.4873147       7      yes
591   591 response 1.2054476 0.07523205 1.0666570 1.3622972       7      yes
592   592 response 2.2987358 0.12851060 2.0601684 2.5649293      12      yes
593   593 response 2.2299485 0.12427095 1.9992124 2.4873147       7      yes
594   594 response 1.2426320 0.07826874 1.0983191 1.4059068      12      yes
595   595 response 1.5650294 0.08231414 1.4117337 1.7349709       3      yes
596   596 response 2.2987358 0.12851060 2.0601684 2.5649293       7      yes
597   597 response 0.8141029 0.07692276 0.6764741 0.9797323       1      yes
598   598 response 1.5181975 0.07908880 1.3708372 1.6813985       7      yes
599   599 response 1.5650294 0.08231414 1.4117337 1.7349709       2      yes
600   600 response 1.2426320 0.07826874 1.0983191 1.4059068       2      yes
601   601 response 2.2299485 0.12427095 1.9992124 2.4873147       1      yes
    gender yearsmarried
1     male       10.000
2   female        4.000
3   female       15.000
4     male       15.000
5     male        0.750
6   female        1.500
7   female        0.750
8     male       15.000
9   female       15.000
10    male        1.500
11    male       15.000
12    male        4.000
13    male       15.000
14  female        1.500
15  female        4.000
16  female       15.000
17  female       15.000
18  female        0.750
19  female        1.500
20  female       10.000
21  female        1.500
22  female        1.500
23  female       10.000
24  female       10.000
25    male        4.000
26  female        1.500
27  female        7.000
28    male       15.000
29    male        4.000
30  female        4.000
31    male       15.000
32  female        1.500
33    male        0.417
34  female       15.000
35    male        4.000
36  female        1.500
37  female       15.000
38  female        4.000
39    male        1.500
40  female        0.750
41    male       10.000
42    male       15.000
43  female        0.417
44  female        4.000
45  female        7.000
46    male        4.000
47  female        7.000
48  female       15.000
49    male        1.500
50    male       15.000
51  female        0.750
52    male        7.000
53    male        4.000
54    male       10.000
55    male        4.000
56  female       15.000
57    male        1.500
58  female       15.000
59  female        0.750
60    male       10.000
61  female       15.000
62    male       10.000
63  female        0.750
64    male        4.000
65    male        7.000
66    male       15.000
67    male       10.000
68  female       15.000
69  female        1.500
70  female        1.500
71  female        4.000
72  female       10.000
73  female        0.125
74    male       15.000
75    male       15.000
76    male        7.000
77  female        1.500
78    male        4.000
79  female        1.500
80    male       15.000
81    male        1.500
82    male       15.000
83  female        0.750
84    male        4.000
85  female        1.500
86  female        0.417
87  female       15.000
88  female        1.500
89  female        1.500
90  female       15.000
91  female       15.000
92    male       10.000
93    male       10.000
94    male       15.000
95    male        0.417
96  female       15.000
97    male       15.000
98    male       10.000
99    male       15.000
100   male       15.000
101 female       10.000
102   male       10.000
103 female        0.125
104   male       15.000
105 female       15.000
106   male        4.000
107   male        7.000
108   male       15.000
109   male       15.000
110 female        1.500
111 female        7.000
112 female       15.000
113 female        1.500
114   male       10.000
115 female        7.000
116   male       15.000
117   male       15.000
118   male       10.000
119 female        0.750
120 female        7.000
121 female        7.000
122 female       15.000
123   male       15.000
124 female        7.000
125 female        1.500
126   male       15.000
127 female        0.125
128   male        1.500
129   male        1.500
130   male        1.500
131 female       10.000
132   male       15.000
133 female        1.500
134   male        4.000
135 female       10.000
136 female       15.000
137 female        0.750
138 female       15.000
139   male        4.000
140   male        4.000
141 female       10.000
142 female       15.000
143   male        7.000
144   male       15.000
145   male        4.000
146   male        4.000
147   male        4.000
148 female       15.000
149 female       15.000
150   male        7.000
151   male        7.000
152   male        0.750
153   male        4.000
154   male       15.000
155   male        0.750
156   male       15.000
157 female       15.000
158 female       15.000
159   male        7.000
160   male        4.000
161 female        4.000
162 female        4.000
163   male       15.000
164 female       15.000
165 female       10.000
166 female        1.500
167 female       15.000
168 female        1.500
169   male       15.000
170   male       15.000
171   male       15.000
172 female        0.125
173 female       10.000
174 female       15.000
175 female        1.500
176   male        0.125
177 female        4.000
178 female       10.000
179 female        7.000
180 female       15.000
181 female       15.000
182   male        1.500
183 female        4.000
184 female        7.000
185 female        0.417
186 female        7.000
187   male        0.750
188   male        4.000
189   male       10.000
190   male       15.000
191   male        0.750
192 female        7.000
193   male        0.417
194   male       15.000
195 female       15.000
196   male        4.000
197   male       15.000
198 female        1.500
199   male       15.000
200 female        1.500
201   male        4.000
202   male        4.000
203 female        1.500
204   male        0.750
205 female        7.000
206 female        0.750
207 female       15.000
208 female        1.500
209 female       10.000
210 female       15.000
211 female       15.000
212   male        4.000
213   male        7.000
214   male        0.750
215 female        4.000
216   male        1.500
217   male        1.500
218   male        4.000
219 female        4.000
220   male       15.000
221 female       10.000
222   male       15.000
223 female       15.000
224   male        1.500
225 female       10.000
226 female        7.000
227 female       10.000
228   male        1.500
229   male       15.000
230   male        4.000
231 female       15.000
232 female        7.000
233 female       15.000
234   male        4.000
235   male        0.750
236   male        4.000
237 female        0.750
238 female       15.000
239   male       10.000
240 female       15.000
241   male        7.000
242 female       15.000
243   male       15.000
244   male        4.000
245 female       15.000
246   male        0.750
247 female        1.500
248 female       15.000
249 female       15.000
250   male       15.000
251 female       15.000
252   male       15.000
253   male        7.000
254 female        7.000
255   male        1.500
256 female        4.000
257 female        1.500
258   male       15.000
259   male       15.000
260 female        7.000
261 female        4.000
262   male       15.000
263   male        7.000
264 female        1.500
265 female        1.500
266   male       15.000
267 female       15.000
268 female       10.000
269   male       15.000
270 female        4.000
271   male       15.000
272   male        1.500
273 female        0.125
274   male       10.000
275 female        4.000
276 female        7.000
277   male        4.000
278 female       15.000
279   male       15.000
280   male        1.500
281   male       15.000
282 female       15.000
283 female        7.000
284 female       15.000
285 female        7.000
286   male        1.500
287   male        1.500
288   male        4.000
289 female        7.000
290 female       15.000
291 female        7.000
292   male        1.500
293   male       15.000
294 female        1.500
295 female       10.000
296   male        4.000
297 female        0.417
298 female        4.000
299   male       15.000
300   male       10.000
301 female        7.000
302   male        4.000
303   male        4.000
304   male        1.500
305 female        4.000
306 female        0.750
307   male       10.000
308 female        0.750
309   male       15.000
310   male        4.000
311 female        1.500
312 female        7.000
313   male       15.000
314   male        4.000
315 female       15.000
316   male        4.000
317 female        4.000
318   male       15.000
319 female        1.500
320   male       15.000
321 female        0.417
322 female        1.500
323   male        4.000
324 female       15.000
325 female        1.500
326   male        4.000
327   male       15.000
328 female       10.000
329 female       10.000
330   male        1.500
331   male        1.500
332 female       10.000
333 female       15.000
334 female       10.000
335   male        4.000
336 female       15.000
337 female        0.750
338 female       15.000
339 female       15.000
340 female       10.000
341   male        7.000
342 female       15.000
343   male        1.500
344 female        0.750
345   male        4.000
346   male        0.125
347   male        1.500
348   male        7.000
349 female       15.000
350   male        1.500
351 female        1.500
352 female        1.500
353   male        0.125
354 female        4.000
355 female        4.000
356 female       15.000
357 female       15.000
358   male        7.000
359   male        0.750
360   male        0.125
361   male       10.000
362 female        0.417
363 female       15.000
364 female       10.000
365   male       15.000
366   male        4.000
367 female        7.000
368 female       10.000
369 female       10.000
370 female        4.000
371 female        7.000
372   male       15.000
373   male        7.000
374 female        1.500
375   male        1.500
376 female        1.500
377 female        7.000
378 female       15.000
379 female        0.750
380 female        1.500
381 female        4.000
382   male       15.000
383   male       10.000
384   male       15.000
385 female        7.000
386 female        1.500
387 female       10.000
388 female        0.750
389 female        1.500
390 female       15.000
391 female        7.000
392   male       15.000
393 female       15.000
394   male       15.000
395 female        7.000
396   male        4.000
397 female        1.500
398 female        0.750
399 female       15.000
400   male        1.500
401   male        4.000
402 female        4.000
403   male       15.000
404   male        1.500
405   male       15.000
406   male        7.000
407   male       15.000
408   male       15.000
409 female        7.000
410 female        7.000
411 female        4.000
412   male        7.000
413   male        7.000
414   male       10.000
415 female        1.500
416 female        4.000
417 female       15.000
418   male       15.000
419 female        7.000
420 female       10.000
421   male        4.000
422 female        4.000
423 female       15.000
424 female        1.500
425   male        7.000
426   male        4.000
427   male        0.125
428 female        7.000
429 female       15.000
430   male       10.000
431 female       15.000
432 female        1.500
433   male        4.000
434 female       15.000
435 female        7.000
436 female        7.000
437 female       15.000
438 female       15.000
439   male        7.000
440   male        7.000
441 female       15.000
442   male        4.000
443 female        7.000
444 female        1.500
445 female        4.000
446 female        4.000
447 female        1.500
448   male       15.000
449   male       10.000
450   male       15.000
451 female        4.000
452   male        1.500
453 female        4.000
454   male       15.000
455 female       10.000
456   male        0.125
457 female        1.500
458   male       15.000
459 female        1.500
460   male       15.000
461 female       15.000
462 female       15.000
463 female       15.000
464 female       15.000
465   male       10.000
466 female       15.000
467   male        4.000
468   male       10.000
469 female        4.000
470   male       15.000
471 female       15.000
472 female        4.000
473 female        7.000
474   male        1.500
475 female        7.000
476 female       15.000
477 female       10.000
478   male        1.500
479   male        4.000
480 female        7.000
481 female       10.000
482   male        4.000
483 female        0.750
484 female       10.000
485 female        7.000
486   male       15.000
487 female       10.000
488 female       10.000
489   male       15.000
490 female       15.000
491   male       15.000
492   male       15.000
493 female        4.000
494   male        7.000
495   male        4.000
496   male       15.000
497 female       15.000
498   male        4.000
499 female        7.000
500   male        0.417
501   male       15.000
502   male       15.000
503   male        4.000
504   male        4.000
505 female       15.000
506   male        4.000
507 female       10.000
508   male        7.000
509   male        7.000
510   male        1.500
511   male        4.000
512   male       15.000
513 female       15.000
514 female        4.000
515   male        4.000
516   male        7.000
517   male       15.000
518 female       15.000
519   male       10.000
520   male       15.000
521 female       15.000
522 female        7.000
523   male        7.000
524   male        4.000
525   male        7.000
526 female       15.000
527 female       15.000
528 female        7.000
529 female        7.000
530   male        1.500
531 female       15.000
532   male       10.000
533   male        4.000
534 female        4.000
535 female       15.000
536   male       10.000
537   male       10.000
538 female        7.000
539 female        7.000
540   male       10.000
541 female        0.750
542 female       15.000
543 female        7.000
544   male        7.000
545   male        4.000
546 female        1.500
547 female       15.000
548 female       15.000
549   male       15.000
550   male       10.000
551 female       15.000
552   male       15.000
553   male       15.000
554 female       15.000
555   male       15.000
556   male       15.000
557   male       10.000
558   male       15.000
559 female       15.000
560   male       10.000
561   male       15.000
562 female        1.500
563 female       15.000
564   male       15.000
565 female        4.000
566 female       10.000
567 female        4.000
568   male        7.000
569 female        4.000
570 female       15.000
571 female        4.000
572 female        0.750
573 female       15.000
574 female        4.000
575   male        4.000
576 female        1.500
577 female       15.000
578   male        7.000
579   male       15.000
580   male       15.000
581   male        4.000
582   male       15.000
583   male       15.000
584   male        1.500
585   male        4.000
586   male       15.000
587 female       15.000
588   male       15.000
589 female       15.000
590 female       15.000
591 female        7.000
592   male       15.000
593 female       15.000
594   male        7.000
595   male       10.000
596   male       15.000
597   male        1.500
598 female       10.000
599   male       10.000
600   male        7.000
601 female       15.000

Predictions on a user-specified grid

Use the datagrid() function and newdata argument:

predictions(
    mod,
    newdata = datagrid(yearsmarried = c(1, 25)))
  rowid     type predicted  std.error  conf.low conf.high children gender
1     1 response  0.759956 0.07385389 0.6281555 0.9194111      yes female
2     2 response  4.810918 0.54304955 3.8560759 6.0021990      yes female
  yearsmarried affairs
1            1       0
2           25       0

Unspecified variables held at their mean or mode.

predictions(
    mod,
    newdata = datagrid(yearsmarried = range))
  rowid     type predicted  std.error conf.low conf.high children gender
1     1 response 0.7105085 0.07318095 0.580627 0.8694434      yes female
2     2 response 2.2299485 0.12427095 1.999212 2.4873147      yes female
  yearsmarried affairs
1        0.125       0
2       15.000       0

Predictions: hypothesis

Two predictions at the range of yearsmarried:

predictions(
    mod,
    newdata = datagrid(yearsmarried = range))
  rowid     type predicted  std.error conf.low conf.high children gender
1     1 response 0.7105085 0.07318095 0.580627 0.8694434      yes female
2     2 response 2.2299485 0.12427095 1.999212 2.4873147      yes female
  yearsmarried affairs
1        0.125       0
2       15.000       0

Are they equal?

predictions(
    mod,
    hypothesis = "b1 = b2",
    newdata = datagrid(yearsmarried = range))
      type  term predicted std.error
1 response b1=b2  -1.51944 0.1407249

Conditional predictions

How does the outcome predicted by my model change as a function of the predictors?

plot_cap(mod, condition = c("yearsmarried", "children"))

Average predictions

predictions(mod) |> summary()
  Predicted Std. Error z value   Pr(>|z|) CI low CI high
1     1.456    0.04935    29.5 < 2.22e-16  1.359   1.553

Model type:  glm 
Prediction type:  response 

Average predictions by subgroup:

predictions(mod, by = "children") |> summary()
  children Predicted Std. Error z value   Pr(>|z|) CI low CI high
1       no    0.9123    0.07303   12.49 < 2.22e-16 0.7691   1.055
2      yes    1.6721    0.06236   26.81 < 2.22e-16 1.5499   1.794

Model type:  glm 
Prediction type:  response 

Predictions on different scales

Response scale (expected counts)

predictions(mod, type = "response") |> head(2)
  rowid     type predicted  std.error  conf.low conf.high affairs children
1     1 response 1.5060531 0.14741307 1.2431523  1.824552       0       no
2     2 response 0.9210563 0.07913696 0.7783073  1.089987       0       no
  gender yearsmarried
1   male           10
2 female            4

Link scale (log)

predictions(mod, type = "link") |> head(2)
  rowid type   predicted  std.error  statistic      p.value   conf.low
1     1 link  0.40949238 0.09788039  4.1835996 2.869293e-05  0.2176503
2     2 link -0.08223412 0.08591978 -0.9571034 3.385150e-01 -0.2506338
   conf.high affairs children gender yearsmarried
1 0.60133442       0       no   male           10
2 0.08616556       0       no female            4

Contrasts

The difference, ratio, or other function of two adjusted predictions, calculated for meaningfully different regressor values (e.g., College graduates vs. Others).

Contrasts are suuuper important

  • What is the difference in predicted cholesterol levels between people in the treatment and control groups?
  • What happens to predicted \(Y\) if \(X\) increases by one standard deviation?
  • Is the estimated probability of conversion higher if the customer sees a blue shirt instead of a red one?

Manual contrasts

If children changes from “no” to “yes”, the expected number of affairs increases by…

# Copy the original data twice
dat_no <- dat_yes <- affairs

# Set different values for the predictor of interest
dat_no$children <- "no"
dat_yes$children <- "yes"

# Predictions
p_no <- predict(mod, newdata = dat_no, type = "response")
p_yes <- predict(mod, newdata = dat_yes, type = "response")

# Compute contrasts for each individual
p_yes - p_no
         1          2          3          4          5          6          7 
0.05897627 0.03606809 0.08403296 0.08662512 0.02895937 0.02976048 0.02809279 
         8          9         10         11         12         13         14 
0.08662512 0.08403296 0.03067850 0.08662512 0.03718069 0.08662512 0.02976048 
        15         16         17         18         19         20         21 
0.03606809 0.08403296 0.08403296 0.02809279 0.02976048 0.05721146 0.02976048 
        22         23         24         25         26         27         28 
0.02976048 0.05721146 0.05721146 0.03718069 0.02976048 0.04542586 0.08662512 
        29         30         31         32         33         34         35 
0.03718069 0.03606809 0.08662512 0.02976048 0.02822729 0.08403296 0.03718069 
        36         37         38         39         40         41         42 
0.02976048 0.08403296 0.03606809 0.03067850 0.02809279 0.05897627 0.08662512 
        43         44         45         46         47         48         49 
0.02738262 0.03606809 0.04542586 0.03718069 0.04542586 0.08403296 0.03067850 
        50         51         52         53         54         55         56 
0.08662512 0.02809279 0.04682711 0.03718069 0.05897627 0.03718069 0.08403296 
        57         58         59         60         61         62         63 
0.03067850 0.08403296 0.02809279 0.05897627 0.08403296 0.05897627 0.02809279 
        64         65         66         67         68         69         70 
0.03718069 0.04682711 0.08662512 0.05897627 0.08403296 0.02976048 0.02976048 
        71         72         73         74         75         76         77 
0.03606809 0.05721146 0.02677467 0.08662512 0.08662512 0.04682711 0.02976048 
        78         79         80         81         82         83         84 
0.03718069 0.02976048 0.08662512 0.03067850 0.08662512 0.02809279 0.03718069 
        85         86         87         88         89         90         91 
0.02976048 0.02738262 0.08403296 0.02976048 0.02976048 0.08403296 0.08403296 
        92         93         94         95         96         97         98 
0.05897627 0.05897627 0.08662512 0.02822729 0.08403296 0.08662512 0.05897627 
        99        100        101        102        103        104        105 
0.08662512 0.08662512 0.05721146 0.05897627 0.02677467 0.08662512 0.08403296 
       106        107        108        109        110        111        112 
0.03718069 0.04682711 0.08662512 0.08662512 0.02976048 0.04542586 0.08403296 
       113        114        115        116        117        118        119 
0.02976048 0.05897627 0.04542586 0.08662512 0.08662512 0.05897627 0.02809279 
       120        121        122        123        124        125        126 
0.04542586 0.04542586 0.08403296 0.08662512 0.04542586 0.02976048 0.08662512 
       127        128        129        130        131        132        133 
0.02677467 0.03067850 0.03067850 0.03067850 0.05721146 0.08662512 0.02976048 
       134        135        136        137        138        139        140 
0.03718069 0.05721146 0.08403296 0.02809279 0.08403296 0.03718069 0.03718069 
       141        142        143        144        145        146        147 
0.05721146 0.08403296 0.04682711 0.08662512 0.03718069 0.03718069 0.03718069 
       148        149        150        151        152        153        154 
0.08403296 0.08403296 0.04682711 0.04682711 0.02895937 0.03718069 0.08662512 
       155        156        157        158        159        160        161 
0.02895937 0.08662512 0.08403296 0.08403296 0.04682711 0.03718069 0.03606809 
       162        163        164        165        166        167        168 
0.03606809 0.08662512 0.08403296 0.05721146 0.02976048 0.08403296 0.02976048 
       169        170        171        172        173        174        175 
0.08662512 0.08662512 0.08662512 0.02677467 0.05721146 0.08403296 0.02976048 
       176        177        178        179        180        181        182 
0.02760059 0.03606809 0.05721146 0.04542586 0.08403296 0.08403296 0.03067850 
       183        184        185        186        187        188        189 
0.03606809 0.04542586 0.02738262 0.04542586 0.02895937 0.03718069 0.05897627 
       190        191        192        193        194        195        196 
0.08662512 0.02895937 0.04542586 0.02822729 0.08662512 0.08403296 0.03718069 
       197        198        199        200        201        202        203 
0.08662512 0.02976048 0.08662512 0.02976048 0.03718069 0.03718069 0.02976048 
       204        205        206        207        208        209        210 
0.02895937 0.04542586 0.02809279 0.08403296 0.02976048 0.05721146 0.08403296 
       211        212        213        214        215        216        217 
0.08403296 0.03718069 0.04682711 0.02895937 0.03606809 0.03067850 0.03067850 
       218        219        220        221        222        223        224 
0.03718069 0.03606809 0.08662512 0.05721146 0.08662512 0.08403296 0.03067850 
       225        226        227        228        229        230        231 
0.05721146 0.04542586 0.05721146 0.03067850 0.08662512 0.03718069 0.08403296 
       232        233        234        235        236        237        238 
0.04542586 0.08403296 0.03718069 0.02895937 0.03718069 0.02809279 0.08403296 
       239        240        241        242        243        244        245 
0.05897627 0.08403296 0.04682711 0.08403296 0.08662512 0.03718069 0.08403296 
       246        247        248        249        250        251        252 
0.02895937 0.02976048 0.08403296 0.08403296 0.08662512 0.08403296 0.08662512 
       253        254        255        256        257        258        259 
0.04682711 0.04542586 0.03067850 0.03606809 0.02976048 0.08662512 0.08662512 
       260        261        262        263        264        265        266 
0.04542586 0.03606809 0.08662512 0.04682711 0.02976048 0.02976048 0.08662512 
       267        268        269        270        271        272        273 
0.08403296 0.05721146 0.08662512 0.03606809 0.08662512 0.03067850 0.02677467 
       274        275        276        277        278        279        280 
0.05897627 0.03606809 0.04542586 0.03718069 0.08403296 0.08662512 0.03067850 
       281        282        283        284        285        286        287 
0.08662512 0.08403296 0.04542586 0.08403296 0.04542586 0.03067850 0.03067850 
       288        289        290        291        292        293        294 
0.03718069 0.04542586 0.08403296 0.04542586 0.03067850 0.08662512 0.02976048 
       295        296        297        298        299        300        301 
0.05721146 0.03718069 0.02738262 0.03606809 0.08662512 0.05897627 0.04542586 
       302        303        304        305        306        307        308 
0.03718069 0.03718069 0.03067850 0.03606809 0.02809279 0.05897627 0.02809279 
       309        310        311        312        313        314        315 
0.08662512 0.03718069 0.02976048 0.04542586 0.08662512 0.03718069 0.08403296 
       316        317        318        319        320        321        322 
0.03718069 0.03606809 0.08662512 0.02976048 0.08662512 0.02738262 0.02976048 
       323        324        325        326        327        328        329 
0.03718069 0.08403296 0.02976048 0.03718069 0.08662512 0.05721146 0.05721146 
       330        331        332        333        334        335        336 
0.03067850 0.03067850 0.05721146 0.08403296 0.05721146 0.03718069 0.08403296 
       337        338        339        340        341        342        343 
0.02809279 0.08403296 0.08403296 0.05721146 0.04682711 0.08403296 0.03067850 
       344        345        346        347        348        349        350 
0.02809279 0.03718069 0.02760059 0.03067850 0.04682711 0.08403296 0.03067850 
       351        352        353        354        355        356        357 
0.02976048 0.02976048 0.02760059 0.03606809 0.03606809 0.08403296 0.08403296 
       358        359        360        361        362        363        364 
0.04682711 0.02895937 0.02760059 0.05897627 0.02738262 0.08403296 0.05721146 
       365        366        367        368        369        370        371 
0.08662512 0.03718069 0.04542586 0.05721146 0.05721146 0.03606809 0.04542586 
       372        373        374        375        376        377        378 
0.08662512 0.04682711 0.02976048 0.03067850 0.02976048 0.04542586 0.08403296 
       379        380        381        382        383        384        385 
0.02809279 0.02976048 0.03606809 0.08662512 0.05897627 0.08662512 0.04542586 
       386        387        388        389        390        391        392 
0.02976048 0.05721146 0.02809279 0.02976048 0.08403296 0.04542586 0.08662512 
       393        394        395        396        397        398        399 
0.08403296 0.08662512 0.04542586 0.03718069 0.02976048 0.02809279 0.08403296 
       400        401        402        403        404        405        406 
0.03067850 0.03718069 0.03606809 0.08662512 0.03067850 0.08662512 0.04682711 
       407        408        409        410        411        412        413 
0.08662512 0.08662512 0.04542586 0.04542586 0.03606809 0.04682711 0.04682711 
       414        415        416        417        418        419        420 
0.05897627 0.02976048 0.03606809 0.08403296 0.08662512 0.04542586 0.05721146 
       421        422        423        424        425        426        427 
0.03718069 0.03606809 0.08403296 0.02976048 0.04682711 0.03718069 0.02760059 
       428        429        430        431        432        433        434 
0.04542586 0.08403296 0.05897627 0.08403296 0.02976048 0.03718069 0.08403296 
       435        436        437        438        439        440        441 
0.04542586 0.04542586 0.08403296 0.08403296 0.04682711 0.04682711 0.08403296 
       442        443        444        445        446        447        448 
0.03718069 0.04542586 0.02976048 0.03606809 0.03606809 0.02976048 0.08662512 
       449        450        451        452        453        454        455 
0.05897627 0.08662512 0.03606809 0.03067850 0.03606809 0.08662512 0.05721146 
       456        457        458        459        460        461        462 
0.02760059 0.02976048 0.08662512 0.02976048 0.08662512 0.08403296 0.08403296 
       463        464        465        466        467        468        469 
0.08403296 0.08403296 0.05897627 0.08403296 0.03718069 0.05897627 0.03606809 
       470        471        472        473        474        475        476 
0.08662512 0.08403296 0.03606809 0.04542586 0.03067850 0.04542586 0.08403296 
       477        478        479        480        481        482        483 
0.05721146 0.03067850 0.03718069 0.04542586 0.05721146 0.03718069 0.02809279 
       484        485        486        487        488        489        490 
0.05721146 0.04542586 0.08662512 0.05721146 0.05721146 0.08662512 0.08403296 
       491        492        493        494        495        496        497 
0.08662512 0.08662512 0.03606809 0.04682711 0.03718069 0.08662512 0.08403296 
       498        499        500        501        502        503        504 
0.03718069 0.04542586 0.02822729 0.08662512 0.08662512 0.03718069 0.03718069 
       505        506        507        508        509        510        511 
0.08403296 0.03718069 0.05721146 0.04682711 0.04682711 0.03067850 0.03718069 
       512        513        514        515        516        517        518 
0.08662512 0.08403296 0.03606809 0.03718069 0.04682711 0.08662512 0.08403296 
       519        520        521        522        523        524        525 
0.05897627 0.08662512 0.08403296 0.04542586 0.04682711 0.03718069 0.04682711 
       526        527        528        529        530        531        532 
0.08403296 0.08403296 0.04542586 0.04542586 0.03067850 0.08403296 0.05897627 
       533        534        535        536        537        538        539 
0.03718069 0.03606809 0.08403296 0.05897627 0.05897627 0.04542586 0.04542586 
       540        541        542        543        544        545        546 
0.05897627 0.02809279 0.08403296 0.04542586 0.04682711 0.03718069 0.02976048 
       547        548        549        550        551        552        553 
0.08403296 0.08403296 0.08662512 0.05897627 0.08403296 0.08662512 0.08662512 
       554        555        556        557        558        559        560 
0.08403296 0.08662512 0.08662512 0.05897627 0.08662512 0.08403296 0.05897627 
       561        562        563        564        565        566        567 
0.08662512 0.02976048 0.08403296 0.08662512 0.03606809 0.05721146 0.03606809 
       568        569        570        571        572        573        574 
0.04682711 0.03606809 0.08403296 0.03606809 0.02809279 0.08403296 0.03606809 
       575        576        577        578        579        580        581 
0.03718069 0.02976048 0.08403296 0.04682711 0.08662512 0.08662512 0.03718069 
       582        583        584        585        586        587        588 
0.08662512 0.08662512 0.03067850 0.03718069 0.08662512 0.08403296 0.08662512 
       589        590        591        592        593        594        595 
0.08403296 0.08403296 0.04542586 0.08662512 0.08403296 0.04682711 0.05897627 
       596        597        598        599        600        601 
0.08662512 0.03067850 0.05721146 0.05897627 0.04682711 0.08403296 

Average contrast

dat_no <- dat_yes <- affairs

dat_no$children <- "no"
dat_yes$children <- "yes"

p_no <- predict(mod, newdata = dat_no, type = "response")
p_yes <- predict(mod, newdata = dat_yes, type = "response")

mean(p_yes - p_no)
[1] 0.05524715

With marginaleffects:

comparisons(mod) |> summary()
          Term      Contrast  Effect Std. Error z value Pr(>|z|)    2.5 %
1     children      yes - no 0.05525    0.14967  0.3691  0.71204 -0.23811
2       gender male - female 0.04425    0.09860  0.4488  0.65357 -0.14900
3 yearsmarried   (x + 1) - x 0.11197    0.01183  9.4635  < 2e-16  0.08878
  97.5 %
1 0.3486
2 0.2375
3 0.1352

Model type:  glm 
Prediction type:  response 

How many additional affairs when yearsmarried increases from 1 to 25?

comparisons(mod, variables = list(yearsmarried = c(1, 25))) |> summary()
          Term Contrast Effect Std. Error z value   Pr(>|z|) 2.5 % 97.5 %
1 yearsmarried   25 - 1  4.066     0.6327   6.427 1.3044e-10 2.826  5.307

Model type:  glm 
Prediction type:  response 

How many additional affairs when yearsmarried increases by 1 SD?

comparisons(mod, variables = list(yearsmarried = "sd")) |> summary()
          Term                Contrast Effect Std. Error z value   Pr(>|z|)
1 yearsmarried (x + sd/2) - (x - sd/2) 0.5718    0.05471   10.45 < 2.22e-16
   2.5 % 97.5 %
1 0.4646 0.6791

Model type:  glm 
Prediction type:  response 

Differences, Ratios, and More…

So far:

\[\hat{Y}_1 - \hat{Y}_0\]

Also:

\[e^{\hat{Y}_1 - \hat{Y}_0}\]

\[\frac{\hat{Y}_1}{\hat{Y}_0}\]

\[ln\left (\frac{\hat{Y}_1}{\hat{Y}_0} \right )\]

\[...\]

Shortcuts:

comparisons(mod, transform_pre = "ratio")
comparisons(mod, transform_pre = "lnratio")

Arbitrary functions:

fun <- \(hi, lo) log(mean(hi) / mean(lo))
comparisons(mod, transform_pre = fun))

Adjusted risk ratios, log odds, etc.

Marginal effects

A partial derivative (slope) of the regression equation with respect to a regressor of interest.

Terminology disaster

  1. “Marginal” = effect of small change (derivative)
  2. “Marginal” = sum or average of unit-level estimates (integral)

Marginal effects

In the simplest linear models, the marginal effect is just the coefficient:

\[y = \alpha + \beta x + \varepsilon\]

\[\frac{\partial y}{\partial x} = \beta\]

In more complicated cases, we need numerical derivatives.

General purpose technology: The slope/derivative measures the “effect” of a small change in \(X\) on \(Y\).

Marginal effects

Linear model with interaction:

mod2 <- lm(mpg ~ hp * wt, data = mtcars)
mfx <- marginaleffects(mod2)
summary(mfx)
  Term   Effect Std. Error z value   Pr(>|z|)    2.5 %  97.5 %
1   hp -0.03051   0.007503  -4.066 4.7815e-05 -0.04521 -0.0158
2   wt -4.13165   0.529558  -7.802 6.0900e-15 -5.16956 -3.0937

Model type:  lm 
Prediction type:  response 

Elasticity:

marginaleffects(mod2, slope = "eyex") |> summary()
  Term Contrast  Effect Std. Error z value Pr(>|z|)   2.5 %    97.5 %
1   hp    eY/eX -0.1447    0.07532  -1.921 0.054679 -0.2924  0.002904
2   wt    eY/eX -0.6264    0.10881  -5.756 8.59e-09 -0.8396 -0.413104

Model type:  lm 
Prediction type:  response 

Conditional marginal effects plot

Interaction means that the “effect” of hp depends on the value of wt:

plot_cme(mod2, effect = "hp", condition = "wt")

Marginal Means

Adjusted predictions of a model, averaged across a “reference grid” of categorical predictors.

Skipping this. Ask me about it in Q&A.

Weird model?

It Just Works™

Bayesian Random Effects

Andrew Heiss:

  • Effect of “Autonomy of Opposition Parties” on “Freedom of the Press”
library(brms)
library(ggplot2)
library(ggdist)

vdem_2015 <- read.csv("https://github.com/vincentarelbundock/marginaleffects/raw/main/data-raw/vdem_2015.csv")

mod <- brm(
  bf(media_index ~ party_autonomy + civil_liberties + (1 | region),
     phi ~ (1 | region)),
  data = vdem_2015,
  family = Beta(),
  control = list(adapt_delta = 0.9))

Posterior predictions

predictions(mod, by = "region") |> summary()
                            region Predicted CI low CI high
1                 Asia and Pacific    0.5915 0.5382  0.6438
2  Eastern Europe and Central Asia    0.6412 0.5993  0.6796
3  Latin America and the Caribbean    0.7517 0.7118  0.7867
4     Middle East and North Africa    0.4407 0.3760  0.5107
5               Sub-Saharan Africa    0.6525 0.6179  0.6837
6 Western Europe and North America    0.9212 0.8986  0.9368

Model type:  brmsfit 
Prediction type:  response 
marginaleffects(mod) |> summary()
             Term     Contrast Effect  2.5 % 97.5 %
1  party_autonomy TRUE - FALSE 0.2029 0.1264 0.2857
2 civil_liberties        dY/dX 0.5688 0.4675 0.6640

Model type:  brmsfit 
Prediction type:  response 

Draws from the posterior

p <- comparisons(mod) |> posteriordraws()

ggplot(p, aes(x = draw, y = contrast)) +
    stat_halfeye() +
    labs(x = "Posterior Distributions", y = "Contrasts")

Generalized Additive Models

library(mgcv)
library(itsadug)

# model with smooths
mod <- bam(Y ~ Group + s(Time, by = Group) + s(Subject, bs = "re"), data = simdat)

plot_cap(mod, condition = "Time")

Robust Standard Errors

Predictions, contrasts or marginal effects classical or heteroskedasticity or cluster-robust standard errors:

mfx <- marginaleffects(mod2, vcov = ~cyl)

summary(mfx)
  Term   Effect Std. Error z value   Pr(>|z|)    2.5 %   97.5 %
1   hp -0.03051   0.005562  -5.485 4.1342e-08 -0.04141 -0.01961
2   wt -4.13165   0.268952 -15.362 < 2.22e-16 -4.65879 -3.60451

Model type:  lm 
Prediction type:  response 

Tables with modelsummary

library(modelsummary)

m1 <- lm(am ~ mpg + hp, data = mtcars)
m2 <- glm(am ~ mpg + hp, family = binomial, data = mtcars)

models <- list("OLS" = m1, "Logit" = m2)

modelsummary(
  models,
  title = "OLS and Logit coefficients are very different.")
OLS and Logit coefficients are very different.
OLS Logit
(Intercept) -1.911 -33.605
(0.550) (15.077)
mpg 0.086 1.260
(0.018) (0.567)
hp 0.004 0.055
(0.002) (0.027)
Num.Obs. 32 32
R2 0.484
R2 Adj. 0.449
AIC 32.1 25.2
BIC 38.0 29.6
Log.Lik. -12.061 -9.616
F 13.610 2.467
RMSE 0.35 0.32

Tables with modelsummary

mfx <- lapply(models, marginaleffects)

modelsummary(
  mfx,
  title = "OLS and Logit marginal effects are pretty similar.")
OLS and Logit marginal effects are pretty similar.
OLS Logit
mpg 0.086 0.124
(0.018) (0.028)
hp 0.004 0.005
(0.002) (0.002)
Num.Obs. 32 32
R2 0.484
R2 Adj. 0.449
AIC 32.1 25.2
BIC 38.0 29.6
Log.Lik. -12.061 -9.616
F 13.610 2.467
RMSE 0.35 0.32

modelsummary

20,000+ words docs & case studies

Thanks to…

  • Dependencies:
    • R
    • data.table: ultra fast and powerful
    • checkmate: user-input checks
    • insight: standardized “extractors” for any model
  • Contributors:
    • Marco Mendoza Aviña, Marcio Augusto Diniz, Resul Umit, Noah Greifer, Philippe Joly, Andrew Heiss, Tyson Barrett, Grant McDermott
  • You!!

https://vincentarelbundock.github.io/marginaleffects